00010000:00010000:2.0:1789302552.170354:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7200/0x64fe43095184b87e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa12 expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.170386:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7200/0x64fe43095184b87e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa12 expref: 90 pid: 5845 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302552.170403:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371031424:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302552.170561:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f123a6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302552.173060:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371031424:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302552.173101:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184b88c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x1d:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.173795:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184b88c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1d:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.173804:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184b88c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x1d:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302552.173827:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1d:0x0]: returning size 1516 00010000:00010000:2.0:1789302552.174027:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184b885 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x17:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa19 expref: 91 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.174045:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184b88c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x1d:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa19 expref: 90 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.174053:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1d:0x0].0x0 (ffff960f02dfd200) 00010000:00010000:2.0:1789302552.174056:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1789302552.174059:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fbc00, rc 0) 00000100:00100000:2.0:1789302552.174104:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:15330:x1876219350140288:12345-192.168.203.11@tcp:101:cp.0 Request processed in 4021us (4233us total) trans 4294967653 rc 0/0 00000100:00100000:2.0:1789302552.174120:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 470 00000100:00100000:2.0:1789302552.195071:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350140800 00000100:00100000:2.0:1789302552.195086:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350140800 00002000:00010000:2.0:1789302552.195137:0:6700:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/10 (0->18446744073709551615) 00002000:00100000:2.0:1789302552.195143:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6aca80 with x1876219350140800 ext(0->18446744073709551615) 00002000:00010000:2.0:1789302552.195159:0:6700:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1e6aca80. 00000100:00100000:2.0:1789302552.195197:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 43 00000100:00100000:2.0:1789302552.195209:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350140800:12345-192.168.203.11@tcp:10:cp.0 00002000:00010000:2.0:1789302552.196246:0:6700:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1789302552.196253:0:6700:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb mtime from disk: 1645827007 -> 1789302551 00002000:00010000:2.0:1789302552.196256:0:6700:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb atime from disk: 1789302175 -> 1789302175 00002000:00010000:2.0:1789302552.196258:0:6700:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb ctime from disk: 1789302551 -> 1789302551 00002000:00010000:2.0:1789302552.196261:0:6700:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xa:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789302552.196352:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350140800:12345-192.168.203.11@tcp:10:cp.0 Request processed in 1141us (2009us total) trans 4294967322 rc 0/0 00000100:00100000:2.0:1789302552.196362:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 43 00002000:00010000:2.0:1789302552.196366:0:6700:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0001: refresh locks: 10737419264/10 (0->18446744073709551615) 00002000:00100000:2.0:1789302552.196369:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6aca80 with x1876219350140800 ext(0->18446744073709551615) 00002000:00010000:2.0:1789302552.196375:0:6700:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1e6aca80. 00000100:00100000:2.0:1789302552.214819:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350141312 00000100:00100000:2.0:1789302552.214832:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350141312 00000100:00100000:2.0:1789302552.214911:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 109 00000100:00100000:2.0:1789302552.214923:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:15330:x1876219350141312:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302552.215331:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:15330:x1876219350141312:12345-192.168.203.11@tcp:35:cp.0 Request processed in 409us (657us total) trans 4294967657 rc 0/0 00000100:00100000:2.0:1789302552.215357:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 109 00000100:00100000:2.0:1789302552.243108:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302552.258549:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350141824 00000100:00100000:2.0:1789302552.258557:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350141824 00000100:00100000:2.0:1789302552.258606:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 11 00000100:00100000:2.0:1789302552.258614:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:2345:x1876219350141824:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302552.258627:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302552.258656:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184b8c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa27 expref: 97 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.258679:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1d:0x0].0x0 (ffff960f02dfd200) 00010000:00010000:2.0:1789302552.258686:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302552.258720:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2345:x1876219350141824:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 107us (4333us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302552.258728:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 11 00000100:00100000:2.0:1789302552.314890:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302552.334991:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302552.356674:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350143360 00000100:00100000:2.0:1789302552.356688:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350143360 00000100:00100000:2.0:1789302552.356756:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 479 00000100:00100000:2.0:1789302552.356767:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:15338:x1876219350143360:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302552.356809:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302552.356836:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb600/0x64fe43095184b93b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fa51 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.357105:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb600/0x64fe43095184b93b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa51 expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.357139:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb600/0x64fe43095184b93b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa51 expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.357163:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff960f11d36b00) 00010000:00010000:2.0:1789302552.357167:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302552.357171:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fb600, rc 0) 00000100:00100000:2.0:1789302552.357441:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:15338:x1876219350143360:12345-192.168.203.11@tcp:101:cp.0 Request processed in 675us (1220us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302552.357448:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 479 00010000:00010000:1.0:1789302552.397834:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184b965 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa58 expref: 104 pid: 5843 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302552.397878:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184b965 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa58 expref: 104 pid: 5843 timeout: 489 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302552.397906:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff960f11d36b00) 00010000:00010000:1.0:1789302552.397911:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184b973 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302552.397923:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184b973 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302552.397933:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184b973 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302552.397957:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302552.398310:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350144896:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 501us (729us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302552.398319:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 162 00000100:00100000:1.0:1789302552.403723:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302552.477148:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 484 00000100:00100000:2.0:1789302552.477165:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:15340:x1876219350145408:12345-192.168.203.11@tcp:101:multiop.0 00010000:00010000:2.0:1789302552.477200:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302552.477233:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184b97a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fa5f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477449:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477466:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477478:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477494:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477540:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477547:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477631:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb200/0x64fe43095184b98f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477640:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb200/0x64fe43095184b98f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477648:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb200/0x64fe43095184b98f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302552.477660:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x1e:0x0]: returning size 1516 00010000:00010000:2.0:1789302552.477793:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477801:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477808:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477815:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fae00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302552.477819:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477827:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477851:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x0 (ffff960f09ae1900) 00010000:00010000:2.0:1789302552.477855:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302552.477861:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477869:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b981 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477884:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477890:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477895:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477901:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fbe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302552.477903:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477909:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477917:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x17:0x0].0x193a76c9 (ffff960f11d36300) 00010000:00010000:2.0:1789302552.477919:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302552.477922:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.477928:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184b988 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x17:0x0].0x193a76c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.478027:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184b97a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x17:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa5f expref: 107 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.478045:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb200/0x64fe43095184b98f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa5f expref: 106 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302552.478053:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x1e:0x0].0x0 (ffff960f11d36b00) 00010000:00010000:2.0:1789302552.478056:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302552.478059:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fb200, rc 0) 00000100:00100000:2.0:1789302552.478150:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:15340:x1876219350145408:12345-192.168.203.11@tcp:101:multiop.0 Request processed in 990us (1532us total) trans 4294967667 rc 0/0 00000100:00100000:2.0:1789302552.478158:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 484 00000100:00100000:1.0:1789302552.484408:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350145664 00000100:00100000:1.0:1789302552.484422:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350145664 00000100:00100000:1.0:1789302552.484490:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 113 00000100:00100000:1.0:1789302552.484519:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f07730450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:15340:x1876219350145664:12345-192.168.203.11@tcp:35:multiop.0 00000100:00100000:1.0:1789302552.485074:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f07730450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:15340:x1876219350145664:12345-192.168.203.11@tcp:35:multiop.0 Request processed in 573us (993us total) trans 4294967668 rc 0/0 00000100:00100000:1.0:1789302552.485082:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 113 00000100:00100000:1.0:1789302552.505629:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000001:00080000:2.0:1789302552.665656:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967320 is committed 00000001:00080000:2.0:1789302552.665682:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967321 is committed 00010000:00010000:2.0:1789302553.230007:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302554.234608:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967321 is committed 00000001:00080000:1.0:1789302554.234636:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967319 is committed 00000001:00080000:1.0:1789302554.234643:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967322 is committed 00000001:00080000:2.0:1789302554.235158:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967669 is committed 00010000:00010000:2.0:1789302554.262274:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302555.277936:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302555.662337:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371032064:0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.662435:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302555.662535:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371032576:0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.662573:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302555.662612:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302555.662621:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371032448:0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.662655:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302555.662694:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371032192:0@lo:400:kworker.0 00000100:00100000:1.0:1789302555.662862:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371032576 00000100:00100000:1.0:1789302555.662868:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371032576 00000100:00100000:1.0:1789302555.662935:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1789302555.662943:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371032576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302555.662993:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371032576:12345-0@lo:400:kworker.0 Request processed in 51us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302555.663001:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:2.0:1789302555.663653:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302555.663687:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371032704:0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.663722:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302555.663744:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302555.663761:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371032320:0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.663784:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302555.663826:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371032448 00000100:00100000:2.0:1789302555.663835:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371032448 00000100:00100000:2.0:1789302555.663890:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371032704 00000100:00100000:2.0:1789302555.663892:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371032704 00000100:00100000:2.0:1789302555.663927:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789302555.663935:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371032448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.664109:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371032448:12345-0@lo:400:kworker.0 Request processed in 173us (1452us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302555.664116:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:2.0:1789302555.664134:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:2.0:1789302555.664140:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371032704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.664252:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371032704:12345-0@lo:400:kworker.0 Request processed in 111us (529us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302555.664259:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00080000:2.0:1789302555.664281:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302555.664285:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302555.664316:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967319 gen 2 00000004:00080000:2.0:1789302555.664322:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f7c9180, transno 4294967319 00000100:00100000:2.0:1789302555.664337:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371032704:0@lo:400:kworker.0 00000100:00100000:1.0:1789302555.664572:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371032192 00000100:00100000:1.0:1789302555.664580:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371032192 00000100:00100000:1.0:1789302555.664623:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371032320 00000100:00100000:1.0:1789302555.664628:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371032320 00000100:00100000:1.0:1789302555.664667:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:1.0:1789302555.664677:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371032192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302555.664744:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371032192:12345-0@lo:400:kworker.0 Request processed in 68us (1089us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302555.664760:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:1.0:1789302555.664775:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:1.0:1789302555.664781:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371032320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302555.664819:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371032320:12345-0@lo:400:kworker.0 Request processed in 27us (1022us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302555.665025:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00080000:1.0:1789302555.665071:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302555.665099:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371032320:0@lo:400:kworker.0 00000100:00100000:1.0:1789302555.665112:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371032192:0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.665137:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371032448:0@lo:400:kworker.0 00000100:00100000:2.0:1789302555.665167:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371032064:0@lo:400:kworker.0 00000100:00080000:1.0:1789302555.925080:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302555.925257:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302555.925294:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302555.925348:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302555.925380:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302555.925443:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350146304 00000100:00100000:1.0:1789302555.925450:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350146304 00000100:00100000:1.0:1789302555.925560:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 203 00000100:00100000:1.0:1789302555.925569:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+47:2348:x1876219350146304:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302555.927648:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302555.927709:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350146176 00000100:00100000:2.0:1789302555.927721:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350146176 00000100:00100000:2.0:1789302555.927792:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:2.0:1789302555.927804:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350146176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302555.927895:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350146176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 93us (2548us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302555.927904:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:2.0:1789302555.927952:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 155 00000100:00100000:2.0:1789302555.927962:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350146560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302555.927993:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350146560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (2699us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302555.927999:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 155 00000100:00100000:2.0:1789302555.928022:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350146688 00000100:00100000:2.0:1789302555.928026:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350146688 00000100:00100000:2.0:1789302555.928063:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 204 00000100:00100000:2.0:1789302555.928070:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2348:x1876219350146688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302555.928207:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2348:x1876219350146688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 136us (570us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302555.928228:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 204 00000100:00080000:2.0:1789302555.928799:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302555.928807:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302555.928962:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+47:2348:x1876219350146304:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 3391us (3702us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302555.928974:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 203 00000100:00080000:1.0:1789302555.933092:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302555.936201:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302555.936722:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350146944 00000100:00100000:1.0:1789302555.936738:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350146944 00000100:00100000:1.0:1789302555.937126:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 121 00000100:00100000:1.0:1789302555.937138:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+5:2344:x1876219350146944:12345-192.168.203.11@tcp:9:kworker.0 00000020:00000080:1.0:1789302555.937180:0:6696:0:(genops.c:1467:class_disconnect()) disconnect: cookie 0x64fe43095184ab49 00000100:00080000:1.0:1789302555.937191:0:6696:0:(import.c:65:import_set_state_nolock()) ffff960f1d467800 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1789302555.937258:0:6696:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967322 00000100:00100000:1.0:1789302555.937841:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+2:2344:x1876219350146944:12345-192.168.203.11@tcp:9:kworker.0 Request processed in 700us (1653us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302555.937855:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 121 00010000:00010000:2.0:1789302555.938341:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302556.174845:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302556.174883:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371032832:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302556.174980:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302556.176080:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371032832:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302556.687168:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371032960 00000100:00100000:1.0:1789302556.687185:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371032960 00000100:00100000:1.0:1789302556.687267:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 89 00000100:00100000:1.0:1789302556.687279:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371032960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302556.687454:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302556.687590:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371032960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 313us (1960us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302556.687598:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 89 00010000:00010000:2.0:1789302557.332333:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1789302557.793079:0:6719:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 584f7d4c-528a-4a63-ad91-f53185ae8e19 00000020:00000080:2.0:1789302557.793088:0:6719:0:(genops.c:885:class_export_put()) final put ffff960f3f50e800/584f7d4c-528a-4a63-ad91-f53185ae8e19 00010000:00010000:2.0:1789302558.349690:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1789302559.041062:0:14336:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 4: Useless cancel must not be registered ========================================================== 08:29:17 (1789302557) 00000100:00100000:2.0:1789302559.136774:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350147328 00000100:00100000:2.0:1789302559.136784:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350147328 00000100:00100000:2.0:1789302559.136855:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:2.0:1789302559.136866:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:15914:x1876219350147328:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:2.0:1789302559.136920:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302559.136947:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095184b9c0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa6d expref: 16 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.136967:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72700) 00010000:00010000:2.0:1789302559.136973:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302559.137176:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b9c7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.137192:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184b4a3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f901 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.137201:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184b9c7 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.137470:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184b4a3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f901 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.137485:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184b4a3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f901 expref: 46 pid: 5845 timeout: 496 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302559.137532:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db47800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371033088:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302559.137619:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f4214e540 going to sleep for 11 seconds 00010000:00010000:2.0:1789302559.163706:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f6e00/0x64fe43095184b9ce lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.163724:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f6e00/0x64fe43095184b9ce lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.163741:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f6e00/0x64fe43095184b9ce lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.163748:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6f6e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302559.163752:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f6e00/0x64fe43095184b9ce lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.163760:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f6e00/0x64fe43095184b9ce lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.163777:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb7cfe2e1 (ffff960f02dfd900) 00010000:00010000:2.0:1789302559.163780:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302559.163785:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f6e00/0x64fe43095184b9ce lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.163869:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f6e00/0x64fe43095184b9ce lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb7cfe2e1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302559.188989:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302559.208158:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350147840 00000100:00100000:1.0:1789302559.208168:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350147840 00000100:00100000:1.0:1789302559.208224:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00100000:1.0:1789302559.208236:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:15918:x1876219350147840:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302559.208258:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302559.208279:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184b9ff lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fa82 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208342:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 3/1,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208356:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208367:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208418:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208427:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208447:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208535:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208543:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208550:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208555:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302559.208560:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208567:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208579:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x0 (ffff960f14168e00) 00010000:00010000:1.0:1789302559.208581:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302559.208586:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208593:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184ba06 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208604:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208612:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208625:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208631:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302559.208634:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208641:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208649:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x9fe5cc27 (ffff960f421e9500) 00010000:00010000:1.0:1789302559.208652:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302559.208655:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208660:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184ba0d lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208692:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184b9ff lrc: 1/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa82 expref: 19 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208702:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x0 (ffff960f14168e00) 00010000:00010000:1.0:1789302559.208705:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302559.208708:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184b9ff lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa82 expref: 19 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.208717:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99600, rc 0) 00000100:00100000:1.0:1789302559.208784:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:15918:x1876219350147840:12345-192.168.203.11@tcp:101:rm.0 Request processed in 550us (723us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302559.208809:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00100000:1.0:1789302559.222442:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302559.313638:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350148352 00000100:00100000:2.0:1789302559.313650:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350148352 00000100:00100000:2.0:1789302559.313695:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 122 00000100:00100000:2.0:1789302559.313704:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:0+-99:2344:x1876219350148352:12345-192.168.203.11@tcp:8:dd.0 00010000:00080000:2.0:1789302559.313749:0:6697:0:(ldlm_lib.c:1453:target_handle_connect()) lustre-OST0001: connection from 584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp t0 exp 0000000000000000 cur 396 last 0 00000020:00000080:2.0:1789302559.313783:0:6697:0:(genops.c:1384:class_connect()) connect: client 584f7d4c-528a-4a63-ad91-f53185ae8e19, cookie 0x64fe43095184ba4c 00002000:00100000:2.0:1789302559.313801:0:6697:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 584f7d4c-528a-4a63-ad91-f53185ae8e19/ffff960f3ff85000 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113a27 ocd_grant: 5111808 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789302559.313825:0:6697:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789302559.313987:0:6697:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789302559.314018:0:6697:0:(import.c:65:import_set_state_nolock()) ffff960f3ff84000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789302559.314083:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2344:x1876219350148352:12345-192.168.203.11@tcp:8:dd.0 Request processed in 378us (942us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302559.314091:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 122 00000001:00100000:2.0:1789302559.356920:0:7654:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 584f7d4c-528a-4a63-ad91-f53185ae8e19 00000020:00000080:2.0:1789302559.356927:0:7654:0:(genops.c:885:class_export_put()) final put ffff960f1d464800/584f7d4c-528a-4a63-ad91-f53185ae8e19 00000100:00100000:2.0:1789302559.361330:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302559.374521:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1789302559.475901:0:7654:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 584f7d4c-528a-4a63-ad91-f53185ae8e19 00000100:00100000:2.0:1789302559.488045:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350148608:12345-192.168.203.11@tcp:4:dd.0 Request processed in 126170us (126709us total) trans 4294967323 rc 0/0 00000100:00100000:2.0:1789302559.488064:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 44 00002000:00100000:2.0:1789302559.488080:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xb:0x0] (0->1048575) req@ffff960f1db46680 x1876219350148608/t4294967323(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:135/0 lens 488/448 e 0 to 0 dl 1789302570 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302559.488104:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1db46680 with x1876219350148608 ext(0->1048575) 00002000:00010000:2.0:1789302559.488112:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1db46680 00000100:00100000:1.0:1789302559.493732:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350148736 00000100:00100000:1.0:1789302559.493747:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350148736 00000100:00100000:1.0:1789302559.493837:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 494 00000100:00100000:1.0:1789302559.493849:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:15921:x1876219350148736:12345-192.168.203.11@tcp:44:dd.0 00000001:00080000:2.0:1789302559.605464:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967671 is committed 00000100:00100000:2.0:1789302559.608852:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302559.608987:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350148864 00000100:00100000:1.0:1789302559.609004:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350148864 00000100:00100000:1.0:1789302559.609266:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 124 00000100:00100000:1.0:1789302559.609280:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350148864:12345-192.168.203.11@tcp:16:dd.0 00000100:00100000:2.0:1789302559.694353:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2347:x1876219350148864:12345-192.168.203.11@tcp:16:dd.0 Request processed in 85069us (85505us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302559.694764:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 124 00000100:00100000:2.0:1789302559.703655:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000001:00080000:1.0:1789302559.704849:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967672 00000100:00100000:1.0:1789302559.704971:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:15921:x1876219350149120:12345-192.168.203.11@tcp:35:dd.0 Request processed in 1125us (1324us total) trans 4294967673 rc 0/0 00000100:00100000:1.0:1789302559.704982:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 114 00000020:00080000:1.0:1789302559.704987:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789302559.788089:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302559.799747:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302559.806993:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807013:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807024:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807350:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb400/0x64fe43095184ba99 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807360:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb400/0x64fe43095184ba99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807367:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb400/0x64fe43095184ba99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302559.807405:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xb:0x0]: returning size 1048576 00000004:00200000:2.0:1789302559.807416:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302559.807438:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb400/0x64fe43095184ba99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807450:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb400/0x64fe43095184ba99 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807458:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184ba8b lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807465:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184ba8b lrc: 2/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807472:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184ba8b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807478:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f121dde00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302559.807483:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184ba8b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807491:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184ba8b lrc: 4/0,0 mode: CR/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807544:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x0 (ffff960f14168e00) 00010000:00010000:2.0:1789302559.807548:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302559.807553:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184ba8b lrc: 2/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807559:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dde00/0x64fe43095184ba8b lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807570:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807585:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807590:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807595:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fbc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302559.807598:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807604:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807612:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x9fe5cc27 (ffff960f11d36800) 00010000:00010000:2.0:1789302559.807615:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302559.807617:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807623:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184ba92 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa:0x0].0x9fe5cc27 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807661:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184ba84 lrc: 0/0,0 mode: --/CR res: [0x200000402:0xa:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739faac expref: 50 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807675:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fb400/0x64fe43095184ba99 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739faac expref: 49 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.807683:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff960f14168b00) 00010000:00010000:2.0:1789302559.807685:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302559.807692:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fb400, rc 0) 00000100:00100000:2.0:1789302559.807785:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+49:15924:x1876219350149504:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4214us (8046us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302559.807801:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 497 00000100:00100000:1.0:1789302559.833945:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350149760 00000100:00100000:1.0:1789302559.833956:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350149760 00000100:00100000:1.0:1789302559.834089:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:1.0:1789302559.834101:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+49:15924:x1876219350149760:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302559.834120:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302559.834142:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184baa7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739faba expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.834212:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184baae lrc: 3/1,0 mode: --/PR res: [0x200000402:0xa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.834228:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184baae lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302559.835820:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184baae lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302559.837897:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xa:0x0]: returning size 16384 00000004:00200000:2.0:1789302559.837907:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302559.837942:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184baae lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.837994:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184baa7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739faba expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.838021:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99000/0x64fe43095184baae lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xa:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739faba expref: 50 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302559.838031:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x0 (ffff960f14168e00) 00010000:00010000:2.0:1789302559.838034:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302559.838038:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99000, rc 0) 00000100:00100000:2.0:1789302559.838200:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:15924:x1876219350149760:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4099us (4481us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302559.838210:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:1.0:1789302559.866336:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350150144 00000100:00100000:1.0:1789302559.866347:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350150144 00000100:00100000:1.0:1789302559.866416:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 499 00000100:00100000:1.0:1789302559.866426:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:15924:x1876219350150144:12345-192.168.203.11@tcp:58:lfs.0 00010000:00010000:2.0:1789302560.397658:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302560.781787:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371033984 00000100:00100000:2.0:1789302560.781802:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371033984 00000100:00100000:2.0:1789302560.781872:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371033600 00000100:00100000:2.0:1789302560.781877:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371033600 00000100:00100000:2.0:1789302560.781897:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371033728 00000100:00100000:2.0:1789302560.781903:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371033728 00000100:00100000:2.0:1789302560.781953:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789302560.781965:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371033984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302560.782094:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371033984:12345-0@lo:400:kworker.0 Request processed in 130us (476us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302560.782104:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:2.0:1789302560.782131:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789302560.782146:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371033600:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302560.782193:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371033600:12345-0@lo:400:kworker.0 Request processed in 52us (499us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302560.782202:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789302560.782232:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789302560.782251:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371033728:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302560.782298:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371033728:12345-0@lo:400:kworker.0 Request processed in 53us (485us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302560.782311:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789302561.036577:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350150272 00000100:00100000:2.0:1789302561.036591:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350150272 00000100:00100000:2.0:1789302561.036648:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 317 00000100:00100000:2.0:1789302561.036672:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350150272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302561.036953:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350150272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 294us (767us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302561.036962:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 317 00000100:00100000:2.0:1789302561.037159:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350150528 00000100:00100000:2.0:1789302561.037166:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350150528 00000100:00100000:2.0:1789302561.037555:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 159 00000100:00100000:2.0:1789302561.037773:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2345:x1876219350150528:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302561.037787:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350150656 00000100:00100000:1.0:1789302561.037795:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350150656 00000100:00100000:2.0:1789302561.037846:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2345:x1876219350150528:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 283us (1591us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302561.037872:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 159 00000100:00100000:2.0:1789302561.037887:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350150912 00000100:00100000:2.0:1789302561.037890:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350150912 00000100:00100000:1.0:1789302561.038753:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 160 00000100:00100000:1.0:1789302561.038767:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350150656:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302561.038928:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 161 00000100:00100000:2.0:1789302561.038937:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2345:x1876219350150912:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302561.038960:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350150656:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 63us (1136us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302561.038970:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2345:x1876219350150912:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (1228us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302561.038973:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 160 00000100:00100000:2.0:1789302561.038984:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 161 00000100:00100000:2.0:1789302561.039012:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350150400 00000100:00100000:2.0:1789302561.039018:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350150400 00000100:00100000:2.0:1789302561.039057:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00100000:2.0:1789302561.039064:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:2345:x1876219350150400:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302561.039096:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:2345:x1876219350150400:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (1442us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302561.039102:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00080000:2.0:1789302561.040022:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302561.295026:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371034240 00000100:00100000:1.0:1789302561.295041:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371034240 00000100:00100000:1.0:1789302561.295108:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 90 00000100:00100000:1.0:1789302561.295119:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371034240:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302561.295169:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 51052544 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302561.295267:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371034240:12345-0@lo:13:osp-pre-1-0.0 Request processed in 149us (1216us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302561.295274:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 90 00010000:00010000:2.0:1789302561.422270:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302561.807305:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371034368 00000100:00100000:2.0:1789302561.807317:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371034368 00000100:00100000:2.0:1789302561.807383:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 91 00000100:00100000:2.0:1789302561.807396:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371034368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302561.807446:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302561.807532:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371034368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 137us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302561.807539:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 91 00010000:00010000:2.0:1789302562.444731:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302564.496020:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302564.909883:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967672 is committed 00000001:00080000:2.0:1789302564.909919:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967673 is committed 00000100:00100000:1.0:1789302565.901215:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371034624:0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.901281:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302565.901344:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302565.901367:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371034752:0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.901394:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302565.901451:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371034624 00000100:00100000:1.0:1789302565.901458:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371034624 00000100:00100000:1.0:1789302565.901528:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371034752 00000100:00100000:1.0:1789302565.901532:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371034752 00000100:00100000:1.0:1789302565.901569:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:1.0:1789302565.901577:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371034624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.901625:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371034624:12345-0@lo:400:kworker.0 Request processed in 48us (344us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302565.901631:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:1.0:1789302565.901650:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789302565.901656:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371034752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.901692:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371034752:12345-0@lo:400:kworker.0 Request processed in 36us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302565.901697:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:1.0:1789302565.901735:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371034752:0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.901745:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371034624:0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.902528:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371034880 00000100:00100000:1.0:1789302565.902534:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371034880 00000100:00100000:1.0:1789302565.902583:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1789302565.902590:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371034880:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.902632:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371034880:12345-0@lo:400:kworker.0 Request processed in 43us (232us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302565.902638:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1789302565.902668:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371034880:0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.907247:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371034496 00000100:00100000:1.0:1789302565.907255:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371034496 00000100:00100000:1.0:1789302565.907309:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:1.0:1789302565.907319:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371034496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.907382:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371034496:12345-0@lo:400:kworker.0 Request processed in 64us (347us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302565.907389:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:1.0:1789302565.909069:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371035008:0@lo:400:kworker.0 00000100:00100000:1.0:1789302565.909134:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302565.909259:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371035136:0@lo:400:kworker.0 00000100:00100000:2.0:1789302565.909291:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302565.909327:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371035136 00000100:00100000:2.0:1789302565.909333:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371035136 00000100:00100000:2.0:1789302565.909385:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:2.0:1789302565.909394:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371035136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302565.909443:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371035136:12345-0@lo:400:kworker.0 Request processed in 50us (150us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302565.909450:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:2.0:1789302565.909487:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371035136:0@lo:400:kworker.0 00000100:00080000:2.0:1789302565.909735:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302565.917118:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371035008:0@lo:400:kworker.0 00000100:00100000:2.0:1789302566.163880:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302566.163921:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302566.163954:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302566.163976:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302566.163997:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302566.164021:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302566.164353:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350151552 00000100:00100000:1.0:1789302566.164369:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350151552 00000100:00100000:1.0:1789302566.164455:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350151168 00000100:00100000:1.0:1789302566.164459:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350151168 00000100:00100000:1.0:1789302566.164489:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 213 00000100:00100000:1.0:1789302566.164544:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2345:x1876219350151552:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302566.164810:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2345:x1876219350151552:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 304us (847us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302566.164827:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 213 00000100:00100000:1.0:1789302566.164860:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 214 00000100:00100000:1.0:1789302566.164869:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:2345:x1876219350151168:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302566.164900:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:2345:x1876219350151168:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (923us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302566.164907:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 214 00000100:00100000:1.0:1789302566.165069:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 165 00000100:00080000:2.0:1789302566.165403:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302566.165409:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302566.165412:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302566.165489:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 166 00000100:00100000:2.0:1789302566.165530:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2345:x1876219350151680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302566.165600:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2345:x1876219350151680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 99us (1575us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302566.165631:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 166 00000100:00100000:1.0:1789302566.166830:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350151424:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302566.167240:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350151424:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 2146us (3239us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302566.167249:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 165 00000100:00100000:2.0:1789302566.413742:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371035264 00000100:00100000:2.0:1789302566.413751:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371035264 00000100:00100000:2.0:1789302566.413807:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 92 00000100:00100000:2.0:1789302566.413817:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371035264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302566.413856:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 51052544 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302566.413922:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371035264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 106us (839us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302566.413930:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 92 00010000:00010000:2.0:1789302566.542047:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302566.925204:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302566.925226:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371035392:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302566.925288:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302566.925349:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371035392 00000100:00100000:2.0:1789302566.925356:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371035392 00000100:00100000:2.0:1789302566.925398:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1789302566.925407:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371035392:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302566.925439:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 45940736 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302566.925482:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371035392:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302566.925490:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 93 00000100:00100000:2.0:1789302566.925551:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371035392:0@lo:13:osp-pre-0-0.0 00000001:02000400:1.0:1789302567.059685:0:14585:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 8: Test default archive number ======== 08:29:25 (1789302565) 00000100:00100000:1.0:1789302567.412788:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302567.413392:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350151936 00000100:00100000:2.0:1789302567.413402:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350151936 00000100:00100000:2.0:1789302567.413531:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:2.0:1789302567.413544:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:16525:x1876219350151936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302567.413581:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302567.413603:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184babc lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fac8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.413802:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.413817:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.413831:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.413846:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.413853:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.413860:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414270:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bad1 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414281:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bad1 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414289:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bad1 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302567.414299:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00010000:00010000:2.0:1789302567.414435:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414442:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414449:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414455:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fbe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.414459:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414466:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414478:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:2.0:1789302567.414481:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.414486:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414492:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bac3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414531:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414537:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414542:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414556:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fae00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.414559:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414564:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414571:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff960f11d36d00) 00010000:00010000:2.0:1789302567.414574:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.414577:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.414581:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184baca lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.417385:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184babc lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fac8 expref: 21 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.417416:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bad1 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fac8 expref: 20 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.417426:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302567.417429:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.417433:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa000, rc 0) 00000100:00100000:2.0:1789302567.417794:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16525:x1876219350151936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4251us (5015us total) trans 4294967674 rc 0/0 00000100:00100000:2.0:1789302567.417803:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:2.0:1789302567.425582:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.431236:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350152192 00000100:00100000:1.0:1789302567.431252:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350152192 00000100:00100000:1.0:1789302567.431874:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 12 00000100:00100000:1.0:1789302567.431885:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350152192:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302567.431902:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302567.431927:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bad1 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fac8 expref: 20 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.431949:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:1.0:1789302567.431954:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302567.432550:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350152192:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 140us (3639us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302567.432559:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 12 00000100:00100000:1.0:1789302567.457400:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302567.457540:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350152320 00000100:00100000:2.0:1789302567.457551:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350152320 00000100:00100000:2.0:1789302567.457607:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 502 00000100:00100000:2.0:1789302567.457617:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:16526:x1876219350152320:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302567.457773:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302567.457793:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184badf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739facf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.457859:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.457873:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.457884:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.457899:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.457906:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.457913:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458241:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458249:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458267:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458273:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fbe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.458277:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458284:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458296:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:2.0:1789302567.458299:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.458304:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458309:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bae6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458320:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458326:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458331:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458335:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.458338:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458353:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458361:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc1d08857 (ffff960f11d36d00) 00010000:00010000:2.0:1789302567.458363:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.458365:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458371:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184baed lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458402:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184badf lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739facf expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458412:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:2.0:1789302567.458415:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.458418:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184badf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739facf expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.458428:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fae00, rc 0) 00000100:00100000:2.0:1789302567.458496:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:16526:x1876219350152320:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 881us (1104us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302567.458598:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 502 00000100:00100000:2.0:1789302567.467471:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302567.478186:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302567.479271:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.479279:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.479303:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.479319:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.479339:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302567.480584:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x20:0x0]: returning size 16384 00010000:00010000:1.0:1789302567.480599:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.480608:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.480615:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.480620:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6f7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.480624:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.480631:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.480643:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:1.0:1789302567.480646:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.480651:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.480656:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184baf4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302567.480698:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008bde5381 reply state 00000000990f111f transno 4294967675 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302567.480707:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302567.480781:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+51:16526:x1876219350152576:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 13062us (13317us total) trans 4294967675 rc 0/0 00000100:00100000:1.0:1789302567.480790:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 503 00010000:00010000:2.0:1789302567.485923:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.485959:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.485965:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.485969:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.485973:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.485978:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.485992:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc1d08857 (ffff960f421e9800) 00010000:00010000:2.0:1789302567.485994:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.485998:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.486002:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bafb lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302567.506421:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.506529:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350152832 00000100:00100000:1.0:1789302567.506540:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350152832 00000100:00100000:1.0:1789302567.506607:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 504 00000100:00100000:1.0:1789302567.506618:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+50:16527:x1876219350152832:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302567.506643:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302567.506684:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb02 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fad6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.506752:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bb09 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.506765:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bb09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.506775:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bb09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302567.506809:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302567.506813:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302567.506852:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bb09 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.506898:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb02 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fad6 expref: 52 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.506916:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bb09 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fad6 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.506924:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f421e9500) 00010000:00010000:1.0:1789302567.506927:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302567.506930:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99c00, rc 0) 00000100:00100000:1.0:1789302567.506988:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+51:16527:x1876219350152832:12345-192.168.203.11@tcp:101:bash.0 Request processed in 372us (576us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302567.506995:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 504 00000100:00100000:1.0:1789302567.515185:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350153088 00000100:00100000:1.0:1789302567.515199:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350153088 00000100:00100000:1.0:1789302567.515302:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00100000:1.0:1789302567.515312:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+52:16527:x1876219350153088:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302567.515330:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302567.515349:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb2c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fae4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515408:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515421:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515432:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515459:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515466:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515473:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515564:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515572:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515578:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515584:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.515589:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515596:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515606:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:1.0:1789302567.515610:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.515614:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515620:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb33 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515637:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515642:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515647:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515652:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.515663:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515676:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515684:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff960f421e9200) 00010000:00010000:1.0:1789302567.515686:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.515689:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515694:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb3a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515725:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb2c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fae4 expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515735:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:1.0:1789302567.515737:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.515760:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb2c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fae4 expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.515771:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99400, rc 0) 00000100:00100000:1.0:1789302567.515924:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+52:16527:x1876219350153088:12345-192.168.203.11@tcp:101:bash.0 Request processed in 613us (844us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302567.515932:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00100000:2.0:1789302567.527047:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302567.534958:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350153344 00000100:00100000:2.0:1789302567.534973:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350153344 00000100:00100000:2.0:1789302567.535080:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 508 00000100:00100000:2.0:1789302567.535091:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:16528:x1876219350153344:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302567.535111:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302567.535130:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bb4f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739faf2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535205:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535218:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535229:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535244:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535251:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535258:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535346:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184bb64 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535354:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184bb64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535361:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184bb64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302567.535387:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x20:0x0]: returning size 16384 00000004:00200000:2.0:1789302567.535391:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302567.535408:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184bb64 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535430:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535437:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535443:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535449:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.535453:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535460:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535486:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f421e9500) 00010000:00010000:2.0:1789302567.535489:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.535493:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535499:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bb56 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535535:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535541:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535547:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535557:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.535560:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535567:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535575:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc1d08857 (ffff960f11d36d00) 00010000:00010000:2.0:1789302567.535578:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.535581:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535587:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bb5d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc1d08857 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535615:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bb4f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739faf2 expref: 21 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535627:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184bb64 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739faf2 expref: 20 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.535634:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:2.0:1789302567.535637:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302567.535640:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fbc00, rc 0) 00000100:00100000:2.0:1789302567.535717:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16528:x1876219350153344:12345-192.168.203.11@tcp:101:rm.0 Request processed in 628us (843us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302567.535725:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 508 00000100:00100000:2.0:1789302567.545683:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.551139:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350153600 00000100:00100000:1.0:1789302567.551148:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350153600 00000100:00100000:1.0:1789302567.551209:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00100000:1.0:1789302567.551219:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16528:x1876219350153600:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302567.551239:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302567.551259:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb80 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb00 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551365:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551380:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551392:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551407:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551413:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551419:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551478:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551487:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551493:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551529:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.551534:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551542:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551553:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:1.0:1789302567.551556:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.551562:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551567:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bb87 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551579:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551585:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551590:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551602:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.551605:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551611:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551619:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff960f421e9200) 00010000:00010000:1.0:1789302567.551621:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.551624:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.551630:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bb8e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.552003:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb80 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb00 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.552068:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:1.0:1789302567.552071:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.552075:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bb80 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb00 expref: 21 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.552085:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99400, rc 0) 00000100:00100000:1.0:1789302567.552185:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16528:x1876219350153600:12345-192.168.203.11@tcp:101:rm.0 Request processed in 966us (1212us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302567.552194:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00100000:2.0:1789302567.590092:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.590184:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350153856 00000100:00100000:1.0:1789302567.590208:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350153856 00000100:00100000:1.0:1789302567.590274:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 512 00000100:00100000:1.0:1789302567.590285:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16529:x1876219350153856:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302567.590304:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302567.590335:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bbaa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb0e expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590400:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590414:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590425:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590439:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590446:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590451:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590530:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590539:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590545:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590558:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.590562:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590570:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590584:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:1.0:1789302567.590587:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.590591:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590597:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbb1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590609:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590614:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590619:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590624:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.590626:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590631:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590649:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff960f421e9200) 00010000:00010000:1.0:1789302567.590651:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.590654:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590660:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbb8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590698:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bbaa lrc: 1/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb0e expref: 21 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590708:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:1.0:1789302567.590710:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.590714:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bbaa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb0e expref: 21 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.590723:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99400, rc 0) 00000100:00100000:1.0:1789302567.590835:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16529:x1876219350153856:12345-192.168.203.11@tcp:101:cp.0 Request processed in 551us (751us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302567.590842:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 512 00000100:00100000:2.0:1789302567.604145:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.604240:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350154112 00000100:00100000:1.0:1789302567.604251:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350154112 00000100:00100000:1.0:1789302567.604320:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 513 00000100:00100000:1.0:1789302567.604340:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16529:x1876219350154112:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302567.604360:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302567.604382:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bbbf lrc: 2/0,0 mode: --/CW res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb15 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.604546:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.604561:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.604572:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.604585:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbcd lrc: 3/0,1 mode: --/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.604592:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbcd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.604598:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbcd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302567.605606:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xc:0x0], index 0: 0 00010000:00010000:1.0:1789302567.605880:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98200/0x64fe43095184bbd4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605894:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98200/0x64fe43095184bbd4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605904:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98200/0x64fe43095184bbd4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605940:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605962:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605969:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605975:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302567.605979:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605987:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.605999:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:1.0:1789302567.606002:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302567.606007:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.606013:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bbc6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302567.606027:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ed8f1960 reply state 000000008474a44a transno 4294967676 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bbcd lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302567.606035:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302567.606071:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bbbf lrc: 0/0,0 mode: --/CW res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb15 expref: 23 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.606086:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98200/0x64fe43095184bbd4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb15 expref: 22 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.606104:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff960f421e9a00) 00010000:00010000:1.0:1789302567.606107:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302567.606110:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc98200, rc 0) 00000100:00100000:1.0:1789302567.606232:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:16529:x1876219350154112:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1892us (2096us total) trans 4294967676 rc 0/0 00000100:00100000:1.0:1789302567.606241:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 513 00000100:00100000:2.0:1789302567.612108:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302567.635710:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350154368 00000100:00100000:2.0:1789302567.635720:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350154368 00000100:00100000:2.0:1789302567.635800:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 128 00000100:00100000:2.0:1789302567.635810:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:0:x1876219350154368:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302567.635836:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302567.635855:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb1c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302567.635885:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302567.635903:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=404s v=(0 0) 00010000:00010000:2.0:1789302567.635908:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302567.635916:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.635933:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 7 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.635946:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa800, rc 0) 00000100:00100000:2.0:1789302567.636003:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:0:x1876219350154368:12345-192.168.203.11@tcp:101:cp.0 Request processed in 194us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302567.636010:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 128 00000100:00100000:2.0:1789302567.639142:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.639228:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350154496 00000100:00100000:1.0:1789302567.639246:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350154496 00000100:00100000:1.0:1789302567.639302:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 129 00000100:00100000:1.0:1789302567.639312:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:16529:x1876219350154496:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302567.639328:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302567.639346:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb23 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.639377:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.639385:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302567.639389:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302567.639395:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 4/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 8 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.639428:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 8 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302567.639464:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 8 pid: 6696 timeout: 504 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302567.639478:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371035648:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302567.639605:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e0281b6c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302567.643302:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63e680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371035648:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302567.643358:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 8 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.643380:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99e00, rc 0) 00000100:00100000:2.0:1789302567.643441:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:16529:x1876219350154496:12345-192.168.203.11@tcp:101:cp.0 Request processed in 4130us (4306us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302567.643449:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 129 00000100:00100000:2.0:1789302567.649037:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350154624 00000100:00100000:2.0:1789302567.649047:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350154624 00010000:00010000:2.0:1789302567.649085:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 8 pid: 6696 timeout: 504 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302567.649125:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 165 00000100:00100000:2.0:1789302567.649136:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350154624:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302567.649150:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302567.649174:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 8 pid: 6696 timeout: 504 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.649188:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 8 pid: 6696 timeout: 504 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.649211:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bbe9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb1c expref: 8 pid: 6696 timeout: 504 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.649227:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302567.649317:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302567.649327:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 7 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.649338:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 7 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.649376:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 7 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302567.649389:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef79500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371035776:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302567.649471:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302567.653162:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350154880 00000100:00100000:1.0:1789302567.653171:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350154880 00000100:00100000:1.0:1789302567.653231:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 115 00000100:00100000:1.0:1789302567.653241:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:16529:x1876219350154880:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302567.654380:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef79500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371035776:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302567.654403:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302567.654454:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350154624:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 5320us (5629us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302567.654894:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:16529:x1876219350154880:12345-192.168.203.11@tcp:35:cp.0 Request processed in 1654us (1897us total) trans 4294967677 rc 0/0 00000100:00100000:1.0:1789302567.654903:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 115 00000100:00100000:2.0:1789302567.654985:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 165 00000100:00100000:1.0:1789302567.729135:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302567.729212:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350155008 00000100:00100000:2.0:1789302567.729222:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350155008 00000100:00100000:2.0:1789302567.729277:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 514 00000100:00100000:2.0:1789302567.729287:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+52:16532:x1876219350155008:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302567.729307:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302567.729327:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bbf7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb2a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729395:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 3/1,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729424:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729434:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729449:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729456:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729474:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729588:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bc0c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729596:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bc0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729618:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bc0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302567.729650:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xc:0x0]: returning size 1516 00000004:00200000:2.0:1789302567.729654:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302567.729680:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bc0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729691:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bc0c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729709:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729715:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729722:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729727:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.729731:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729749:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729761:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:2.0:1789302567.729764:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.729770:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729776:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bbfe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729790:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729796:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729801:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729811:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302567.729814:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729820:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729828:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x70ee8377 (ffff960f11d36800) 00010000:00010000:2.0:1789302567.729831:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302567.729833:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729839:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc05 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x70ee8377 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729871:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bbf7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb2a expref: 54 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729884:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa600/0x64fe43095184bc0c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb2a expref: 53 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.729891:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff960f421e9a00) 00010000:00010000:2.0:1789302567.729893:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302567.729895:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa600, rc 0) 00000100:00100000:2.0:1789302567.729954:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+53:16532:x1876219350155008:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 669us (828us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302567.729961:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 514 00000100:00100000:2.0:1789302567.738359:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350155136 00000100:00100000:2.0:1789302567.738383:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350155136 00000100:00100000:2.0:1789302567.738442:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 130 00000100:00100000:2.0:1789302567.738452:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350155136:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302567.738481:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302567.738534:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bc13 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb31 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302567.738595:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef79500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371035904:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302567.738719:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0cc0 going to sleep for 11 seconds 00002000:00010000:2.0:1789302567.741810:0:6697:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb size: 0 -> 1516 00002000:00010000:2.0:1789302567.741819:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb mtime: 0 -> 1789302567 00002000:00010000:2.0:1789302567.741821:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb atime: 0 -> 1789302567 00002000:00010000:2.0:1789302567.741824:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xc:0x0] updating lvb ctime: 0 -> 1789302567 00000100:00100000:2.0:1789302567.741846:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef79500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371035904:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302567.741880:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bc13 lrc: 1/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb31 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.741904:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000df5d205a/0x64fe43095184bc13 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb31 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302567.741917:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa800, rc 0) 00000100:00100000:2.0:1789302567.741972:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350155136:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3522us (3721us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302567.741981:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 130 00000100:00100000:1.0:1789302567.751576:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.762297:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302567.776430:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302567.776617:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350155648 00000100:00100000:2.0:1789302567.776627:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350155648 00000100:00100000:2.0:1789302567.776714:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 516 00000100:00100000:2.0:1789302567.776724:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+54:16532:x1876219350155648:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:2.0:1789302567.777011:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].11, 136 off9688 00000100:00100000:2.0:1789302567.777601:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+54:16532:x1876219350155648:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 875us (1176us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302567.777612:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 516 00000100:00100000:1.0:1789302568.428271:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302568.432833:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302568.432929:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371036288:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302568.432959:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302568.439431:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350156416 00000100:00100000:2.0:1789302568.439441:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350156416 00002000:00100000:2.0:1789302568.439492:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xc:0x0] (0->1515) req@ffff960f3ef51880 x1876219350156416/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:144/0 lens 2008/0 e 0 to 0 dl 1789302579 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302568.444090:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef51880 with x1876219350156416 ext(0->1515) 00010000:00010000:2.0:1789302568.444107:0:6700:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 7 pid: 8571 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302568.444132:0:6700:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 7 pid: 8571 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302568.444141:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3ef51880 00000100:00100000:2.0:1789302568.455202:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350156544 00000100:00100000:2.0:1789302568.455213:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350156544 00010000:00010000:2.0:1789302568.455254:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 9 pid: 8571 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302568.455293:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 166 00000100:00100000:2.0:1789302568.455303:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350156544:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302568.455329:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302568.455364:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 9 pid: 8571 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302568.455379:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 9 pid: 8571 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302568.455403:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000790d8d65/0x64fe43095184bbf0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb23 expref: 9 pid: 8571 timeout: 505 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302568.455417:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302568.455421:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302568.455428:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302568.455438:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302568.455475:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302568.455544:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350156544:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 241us (859us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302568.455551:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 166 00010000:00010000:1.0:1789302568.459141:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459179:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459227:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459237:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459247:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459254:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6f6e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302568.459260:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459268:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459290:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302568.459298:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000efba1413/0x64fe43095184bc44 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302568.459979:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350156160:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 31510us (31714us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302568.459989:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 132 00000100:00100000:2.0:1789302568.481945:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302568.488377:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302568.496943:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302568.523841:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302568.541761:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350157568 00000100:00100000:2.0:1789302568.541772:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350157568 00000100:00100000:2.0:1789302568.541834:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 522 00000100:00100000:2.0:1789302568.541845:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:16542:x1876219350157568:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302568.541905:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xc:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302568.542002:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:16542:x1876219350157568:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 159us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302568.542015:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 522 00000100:00100000:2.0:1789302568.550862:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302568.982027:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:1.0:1789302568.982957:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302569.008473:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350159104 00000100:00100000:2.0:1789302569.008485:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350159104 00000100:00100000:2.0:1789302569.008928:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 117 00000100:00100000:2.0:1789302569.008949:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+55:16545:x1876219350159104:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:2.0:1789302569.009188:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:16545:x1876219350159104:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 241us (1579us total) trans 4294967682 rc 0/0 00000100:00100000:2.0:1789302569.009197:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 117 00000100:00100000:1.0:1789302571.020894:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371036800:0@lo:400:kworker.0 00000100:00100000:1.0:1789302571.020964:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302571.021023:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1789302571.021029:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371036672:0@lo:400:kworker.0 00000100:00100000:1.0:1789302571.021050:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302571.021064:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371037184:0@lo:400:kworker.0 00000100:00100000:1.0:1789302571.021110:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302571.021153:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371037184 00000100:00100000:1.0:1789302571.021159:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371037184 00000100:00100000:1.0:1789302571.021222:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:1.0:1789302571.021232:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371037184:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302571.021282:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371037184:12345-0@lo:400:kworker.0 Request processed in 51us (170us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302571.021289:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00080000:1.0:1789302571.021310:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302571.021337:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371037184:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021457:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371036544:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021493:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302571.021546:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371037056:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021569:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302571.021588:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302571.021593:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371036928:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021615:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302571.021640:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371036800 00000100:00100000:2.0:1789302571.021655:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371036800 00000100:00100000:2.0:1789302571.021689:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371036672 00000100:00100000:2.0:1789302571.021692:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371036672 00000100:00100000:2.0:1789302571.021704:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371037056 00000100:00100000:2.0:1789302571.021706:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371037056 00000100:00100000:2.0:1789302571.021724:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1789302571.021741:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371036800:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021777:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371036800:12345-0@lo:400:kworker.0 Request processed in 38us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302571.021783:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1789302571.021797:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789302571.021802:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371036672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021827:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371036672:12345-0@lo:400:kworker.0 Request processed in 25us (776us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302571.021833:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789302571.021842:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789302571.021848:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371037056:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021877:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371037056:12345-0@lo:400:kworker.0 Request processed in 29us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302571.021882:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789302571.021899:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371036928 00000100:00100000:2.0:1789302571.021903:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371036928 00000100:00100000:2.0:1789302571.021927:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:2.0:1789302571.021933:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371036928:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.021963:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371036928:12345-0@lo:400:kworker.0 Request processed in 30us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302571.021968:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00080000:2.0:1789302571.021987:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302571.021991:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302571.022006:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371036928:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.022015:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371037056:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.022048:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371036672:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.022058:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63ce00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371036800:0@lo:400:kworker.0 00000100:00100000:1.0:1789302571.022235:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371036544 00000100:00100000:1.0:1789302571.022239:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371036544 00000100:00100000:1.0:1789302571.022281:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:1.0:1789302571.022287:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371036544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302571.022329:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371036544:12345-0@lo:400:kworker.0 Request processed in 42us (835us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302571.022335:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00100000:2.0:1789302571.022605:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63c380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371036544:0@lo:400:kworker.0 00000100:00100000:2.0:1789302571.277662:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350159488 00000100:00100000:2.0:1789302571.277674:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350159488 00000100:00100000:2.0:1789302571.277753:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350160000 00000100:00100000:2.0:1789302571.277760:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350160000 00000100:00100000:2.0:1789302571.277789:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350159616 00000100:00100000:2.0:1789302571.277794:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350159616 00000100:00100000:2.0:1789302571.277813:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350159872 00000100:00100000:2.0:1789302571.277815:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350159872 00000100:00100000:2.0:1789302571.277835:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 169 00000100:00100000:2.0:1789302571.277844:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2346:x1876219350159488:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302571.278551:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350159232 00000100:00100000:1.0:1789302571.278557:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350159232 00000100:00100000:1.0:1789302571.278590:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:1.0:1789302571.278597:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350159232:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302571.282802:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2346:x1876219350159488:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 4944us (5180us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302571.282823:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 169 00000100:00100000:2.0:1789302571.282867:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 172 00000100:00100000:2.0:1789302571.282881:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350159872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302571.282931:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350159872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 54us (5140us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302571.282939:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 172 00000100:00080000:2.0:1789302571.283640:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302571.283656:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302571.283660:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302571.284093:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350159232:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5493us (6651us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302571.284101:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:1.0:1789302571.537131:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371037312:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302571.537205:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302571.537379:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371037312 00000100:00100000:2.0:1789302571.537414:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371037312 00000100:00100000:2.0:1789302571.537469:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 94 00000100:00100000:2.0:1789302571.537478:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371037312:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302571.537539:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 51052544 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302571.537596:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371037312:12345-0@lo:13:osp-pre-1-0.0 Request processed in 118us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302571.537603:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 94 00000100:00100000:1.0:1789302571.537817:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371037312:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302572.046074:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302572.046099:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371037440:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302572.046167:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302572.046691:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371037440:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302574.794532:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350160128 00000100:00100000:2.0:1789302574.794543:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350160128 00000100:00100000:2.0:1789302574.794637:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 526 00000100:00100000:2.0:1789302574.794649:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16525:x1876219350160128:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1789302574.794766:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:16525:x1876219350160128:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 118us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302574.794773:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 526 00000100:00100000:2.0:1789302574.799850:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302575.756842:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302576.141990:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371037696 00000100:00100000:2.0:1789302576.141998:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371037696 00000100:00100000:2.0:1789302576.142035:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371037824 00000100:00100000:2.0:1789302576.142038:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371037824 00000100:00100000:2.0:1789302576.142070:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789302576.142078:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371037696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302576.142141:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371037696:12345-0@lo:400:kworker.0 Request processed in 64us (1225us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302576.142160:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:2.0:1789302576.142184:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:2.0:1789302576.142192:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371037824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302576.142240:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371037824:12345-0@lo:400:kworker.0 Request processed in 49us (1170us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302576.142247:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00080000:2.0:1789302576.142269:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302576.143377:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371037568:0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143427:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302576.143463:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371038080:0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143488:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302576.143530:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789302576.143537:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371037952:0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143561:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302576.143590:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371038080 00000100:00100000:1.0:1789302576.143603:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371038080 00000100:00100000:1.0:1789302576.143657:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:1.0:1789302576.143665:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371038080:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143709:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371038080:12345-0@lo:400:kworker.0 Request processed in 44us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.143715:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:1.0:1789302576.143734:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371037568 00000100:00100000:1.0:1789302576.143738:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371037568 00000100:00100000:1.0:1789302576.143761:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:1.0:1789302576.143767:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371037568:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143797:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371037568:12345-0@lo:400:kworker.0 Request processed in 30us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.143802:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:1.0:1789302576.143828:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371037952 00000100:00100000:1.0:1789302576.143832:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371037952 00000100:00100000:1.0:1789302576.143859:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1789302576.143865:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371037952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143895:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371037952:12345-0@lo:400:kworker.0 Request processed in 30us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.143901:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00080000:1.0:1789302576.143916:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302576.143919:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302576.143949:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371037952:0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143957:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371037568:0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.143965:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371038080:0@lo:400:kworker.0 00000100:00100000:1.0:1789302576.404695:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.404748:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.404779:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.404804:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.404826:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.404852:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.404883:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.404907:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350161024 00000100:00100000:1.0:1789302576.404912:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350161024 00000100:00100000:1.0:1789302576.404950:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350160640 00000100:00100000:1.0:1789302576.404954:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350160640 00000100:00100000:1.0:1789302576.404973:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 223 00000100:00100000:1.0:1789302576.404980:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350161024:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302576.410091:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350161024:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5098us (5297us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.410110:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 223 00000100:00100000:1.0:1789302576.410149:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 224 00000100:00100000:1.0:1789302576.410163:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+53:2348:x1876219350160640:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302576.410220:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+53:2348:x1876219350160640:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 62us (5393us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.410226:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 224 00000100:00100000:1.0:1789302576.410258:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 176 00000100:00100000:1.0:1789302576.410266:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350161280:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302576.410294:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350161280:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (5490us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.410300:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 176 00000100:00100000:1.0:1789302576.410326:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:1.0:1789302576.410333:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350160896:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302576.410360:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350160896:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (5513us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.410366:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:1.0:1789302576.410377:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 178 00000100:00100000:1.0:1789302576.410383:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350161152:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302576.410406:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350161152:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (5526us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302576.410412:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 178 00000100:00080000:1.0:1789302576.410444:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302576.410448:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302576.410452:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:02000400:2.0:1789302576.424907:0:14836:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 9A: Use of explicit archive number, with dedicated copytool ========================================================== 08:29:34 (1789302574) 00000100:00100000:1.0:1789302576.910842:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302576.911033:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350161536 00000100:00100000:2.0:1789302576.911043:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350161536 00000100:00100000:2.0:1789302576.911102:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 527 00000100:00100000:2.0:1789302576.911113:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:17151:x1876219350161536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302576.911134:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302576.911159:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bc8a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb62 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.911244:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.911260:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.911282:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.911298:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912060:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912071:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912371:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bc9f lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912526:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bc9f lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912536:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bc9f lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302576.912548:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00010000:00010000:2.0:1789302576.912761:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912770:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912779:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912785:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302576.912790:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912798:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912824:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:2.0:1789302576.912827:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302576.912833:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912840:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bc91 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912854:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912861:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912938:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912946:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302576.912949:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912958:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912967:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff960f11d36300) 00010000:00010000:2.0:1789302576.912970:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302576.912974:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.912981:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bc98 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.913813:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bc8a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb62 expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.913830:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bc9f lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb62 expref: 22 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302576.913838:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302576.913840:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302576.913843:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fbe00, rc 0) 00000100:00100000:2.0:1789302576.914439:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:17151:x1876219350161536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 3325us (3603us total) trans 4294967684 rc 0/0 00000100:00100000:2.0:1789302576.914448:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 527 00000100:00100000:1.0:1789302576.919223:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302576.920457:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302577.165355:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302577.165382:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371038464:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302577.165447:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302577.165841:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371038464:0@lo:13:osp-pre-0-0.0 00000040:00080000:1.0:1789302577.805829:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302577.805842:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302577.806750:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:1.0:1789302577.806768:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302577.809200:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302579.865982:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302580.286339:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967684 is committed 00000100:00100000:1.0:1789302581.269221:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371038592:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269298:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302581.269347:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371039104:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269394:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302581.269421:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789302581.269426:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371038976:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269448:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302581.269473:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371038720:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269494:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302581.269528:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371039232:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269554:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302581.269583:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302581.269589:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371038848:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269609:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302581.269721:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371038592 00000100:00100000:1.0:1789302581.269729:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371038592 00000100:00100000:1.0:1789302581.269773:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1789302581.269782:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371038592:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269829:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371038592:12345-0@lo:400:kworker.0 Request processed in 48us (531us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.269836:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1789302581.269866:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371039104 00000100:00100000:1.0:1789302581.269870:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371039104 00000100:00100000:1.0:1789302581.269897:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371038720 00000100:00100000:1.0:1789302581.269900:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371038720 00000100:00100000:1.0:1789302581.269910:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371038848 00000100:00100000:1.0:1789302581.269912:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371038848 00000100:00100000:1.0:1789302581.269923:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:1.0:1789302581.269928:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371039104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.269958:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371039104:12345-0@lo:400:kworker.0 Request processed in 30us (563us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.269973:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00100000:1.0:1789302581.269986:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:1.0:1789302581.269991:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371038720:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270022:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371038720:12345-0@lo:400:kworker.0 Request processed in 31us (527us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.270028:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:1.0:1789302581.270038:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1789302581.270043:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371038848:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270068:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371038848:12345-0@lo:400:kworker.0 Request processed in 25us (458us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.270073:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1789302581.270089:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371038976 00000100:00100000:1.0:1789302581.270093:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371038976 00000100:00100000:1.0:1789302581.270116:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371039232 00000100:00100000:1.0:1789302581.270119:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371039232 00000100:00100000:1.0:1789302581.270137:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1789302581.270143:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371038976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270172:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371038976:12345-0@lo:400:kworker.0 Request processed in 29us (722us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.270178:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:1.0:1789302581.270188:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:1.0:1789302581.270193:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371039232:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270217:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371039232:12345-0@lo:400:kworker.0 Request processed in 25us (663us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.270222:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:1.0:1789302581.270271:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371038976:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270280:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371038592:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270288:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371039104:0@lo:400:kworker.0 00000100:00080000:1.0:1789302581.270311:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302581.270314:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302581.270316:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302581.270330:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371038848:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270339:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371038720:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.270347:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371039232:0@lo:400:kworker.0 00000100:00100000:1.0:1789302581.518044:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302581.518096:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302581.518128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302581.518161:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302581.518180:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302581.518198:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302581.518221:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302581.518260:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162304 00000100:00100000:1.0:1789302581.518268:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162304 00000100:00100000:1.0:1789302581.518320:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162560 00000100:00100000:1.0:1789302581.518324:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162560 00000100:00100000:1.0:1789302581.518337:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162176 00000100:00100000:1.0:1789302581.518339:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162176 00000100:00100000:1.0:1789302581.518351:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162688 00000100:00100000:1.0:1789302581.518353:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162688 00000100:00100000:1.0:1789302581.518367:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 181 00000100:00100000:1.0:1789302581.518376:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350162304:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302581.518458:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350162304:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 83us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.518466:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 181 00000100:00100000:1.0:1789302581.518482:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:1.0:1789302581.518489:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350162560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302581.518545:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350162560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 56us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.518552:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:1.0:1789302581.518563:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 183 00000100:00100000:1.0:1789302581.518796:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2348:x1876219350162176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302581.518826:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2348:x1876219350162176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 257us (645us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.518832:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 183 00000100:00100000:1.0:1789302581.518843:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 184 00000100:00100000:1.0:1789302581.518849:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350162688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302581.518868:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350162688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (647us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.518873:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 184 00000100:00100000:1.0:1789302581.518891:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350161920 00000100:00100000:1.0:1789302581.518894:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350161920 00000100:00100000:1.0:1789302581.518920:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:1.0:1789302581.518926:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350161920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302581.518949:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350161920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (788us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.518963:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:1.0:1789302581.518998:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162048 00000100:00100000:1.0:1789302581.519002:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162048 00000100:00100000:1.0:1789302581.519027:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162432 00000100:00100000:1.0:1789302581.519030:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162432 00000100:00100000:1.0:1789302581.519051:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 228 00000100:00100000:1.0:1789302581.519058:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+53:2348:x1876219350162048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302581.519089:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+53:2348:x1876219350162048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (1056us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.519096:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 228 00000100:00100000:1.0:1789302581.519108:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00100000:1.0:1789302581.519114:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350162432:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302581.519139:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350162432:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (939us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302581.519145:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00080000:1.0:1789302581.519164:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302581.519169:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302581.519170:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302581.772793:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371039360:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302581.772864:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302582.033794:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302582.033868:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302582.037150:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302582.037249:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162816 00000100:00100000:2.0:1789302582.037259:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162816 00002000:00100000:2.0:1789302582.037310:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x9:0x0] (0->1515) req@ffff960e01149c00 x1876219350162816/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 2008/0 e 0 to 0 dl 1789302593 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.037320:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e01149c00 with x1876219350162816 ext(0->1515) 00002000:00010000:2.0:1789302582.037325:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e01149c00 00000100:00100000:2.0:1789302582.037348:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350162944 00000100:00100000:2.0:1789302582.037350:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350162944 00002000:00100000:2.0:1789302582.037364:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x9:0x0] (0->1518) req@ffff960f3f63f100 x1876219350162944/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 2008/0 e 0 to 0 dl 1789302593 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.037378:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63f100 with x1876219350162944 ext(0->1518) 00002000:00010000:2.0:1789302582.037382:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f63f100 00000100:00100000:2.0:1789302582.037395:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 46 00000100:00100000:2.0:1789302582.037403:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2348:x1876219350162816:12345-192.168.203.11@tcp:4:cat.0 00000100:00100000:2.0:1789302582.037926:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350162816:12345-192.168.203.11@tcp:4:cat.0 Request processed in 521us (4137us total) trans 4294967323 rc 0/0 00000100:00100000:2.0:1789302582.037934:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 46 00002000:00100000:2.0:1789302582.037941:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x9:0x0] (0->1515) req@ffff960e01149c00 x1876219350162816/t4294967323(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 2008/448 e 0 to 0 dl 1789302593 ref 1 fl Complete:/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.037949:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e01149c00 with x1876219350162816 ext(0->1515) 00002000:00010000:2.0:1789302582.037952:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e01149c00 00000100:00100000:2.0:1789302582.037973:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 47 00000100:00100000:2.0:1789302582.037982:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350162944:12345-192.168.203.11@tcp:4:dd.0 00000100:00100000:1.0:1789302582.038412:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302582.038663:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350162944:12345-192.168.203.11@tcp:4:dd.0 Request processed in 681us (4793us total) trans 4294967324 rc 0/0 00000100:00100000:2.0:1789302582.038671:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 47 00002000:00100000:2.0:1789302582.038678:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x9:0x0] (0->1518) req@ffff960f3f63f100 x1876219350162944/t4294967324(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 2008/448 e 0 to 0 dl 1789302593 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.038687:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63f100 with x1876219350162944 ext(0->1518) 00002000:00010000:2.0:1789302582.038691:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f63f100 00000100:00100000:2.0:1789302582.038706:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350163072 00000100:00100000:2.0:1789302582.038711:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350163072 00002000:00100000:2.0:1789302582.038739:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xa:0x0] (0->1515) req@ffff960f3f63c700 x1876219350163072/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 2008/0 e 0 to 0 dl 1789302593 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.038747:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63c700 with x1876219350163072 ext(0->1515) 00002000:00010000:2.0:1789302582.038750:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f63c700 00000100:00100000:2.0:1789302582.038761:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350163200 00000100:00100000:2.0:1789302582.038764:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350163200 00002000:00100000:2.0:1789302582.038776:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (0->646) req@ffff960f3f63c380 x1876219350163200/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 1136/0 e 0 to 0 dl 1789302593 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.038782:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63c380 with x1876219350163200 ext(0->646) 00002000:00010000:2.0:1789302582.038785:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f63c380 00000100:00100000:2.0:1789302582.038793:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 48 00000100:00100000:2.0:1789302582.038800:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350163072:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:2.0:1789302582.039014:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350163072:12345-192.168.203.11@tcp:4:cp.0 Request processed in 213us (1873us total) trans 4294967324 rc 0/0 00000100:00100000:2.0:1789302582.039020:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 48 00002000:00100000:2.0:1789302582.039026:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xa:0x0] (0->1515) req@ffff960f3f63c700 x1876219350163072/t4294967324(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 2008/448 e 0 to 0 dl 1789302593 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.039033:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63c700 with x1876219350163072 ext(0->1515) 00002000:00010000:2.0:1789302582.039036:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f63c700 00000100:00100000:2.0:1789302582.039049:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 49 00000100:00100000:2.0:1789302582.039056:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350163200:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:2.0:1789302582.039257:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350163200:12345-192.168.203.11@tcp:4:cp.0 Request processed in 200us (846us total) trans 4294967325 rc 0/0 00000100:00100000:2.0:1789302582.039264:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 49 00002000:00100000:2.0:1789302582.039277:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xa:0x0] (0->646) req@ffff960f3f63c380 x1876219350163200/t4294967325(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:158/0 lens 1136/448 e 0 to 0 dl 1789302593 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302582.039284:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63c380 with x1876219350163200 ext(0->646) 00002000:00010000:2.0:1789302582.039287:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f63c380 00010000:00010000:1.0:1789302583.952245:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302585.436409:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967325 is committed 00010000:00010000:1.0:1789302585.998341:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302586.383548:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371039616:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.383613:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302586.383654:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371040128:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.383681:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302586.383707:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789302586.383714:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371040000:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.383744:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302586.383788:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371039744 00000100:00100000:1.0:1789302586.383795:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371039744 00000100:00100000:1.0:1789302586.383855:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371039872 00000100:00100000:1.0:1789302586.383861:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371039872 00000100:00100000:1.0:1789302586.383890:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371040128 00000100:00100000:1.0:1789302586.383894:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371040128 00000100:00100000:1.0:1789302586.383917:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1789302586.383927:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371039744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.383987:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371039744:12345-0@lo:400:kworker.0 Request processed in 62us (1406us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302586.383996:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1789302586.384012:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:1.0:1789302586.384021:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371039872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384052:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371039872:12345-0@lo:400:kworker.0 Request processed in 32us (1306us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302586.384059:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:1.0:1789302586.384075:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:1.0:1789302586.384083:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371040128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384121:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371040128:12345-0@lo:400:kworker.0 Request processed in 39us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302586.384140:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00080000:1.0:1789302586.384165:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302586.384170:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302586.384172:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302586.384185:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1789302586.384192:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371040000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384229:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371040000:12345-0@lo:400:kworker.0 Request processed in 37us (486us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302586.384235:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1789302586.384285:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371040000:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384300:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371039616:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384309:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371040128:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384356:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371039872:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384370:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371039744:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.384382:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371040256:0@lo:400:kworker.0 00000100:00100000:1.0:1789302586.639994:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302586.640047:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302586.640182:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350163456 00000100:00100000:2.0:1789302586.640209:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350163456 00000100:00100000:2.0:1789302586.640287:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 233 00000100:00100000:2.0:1789302586.640301:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+53:2347:x1876219350163456:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302586.640409:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+53:2347:x1876219350163456:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 112us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302586.640417:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 233 00000100:00100000:1.0:1789302586.642383:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302586.643481:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302586.643566:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302586.646219:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302586.646269:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302586.646674:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350163584 00000100:00100000:2.0:1789302586.646682:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350163584 00000100:00100000:2.0:1789302586.646740:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 189 00000100:00100000:2.0:1789302586.646749:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350163584:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302586.646980:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350163584:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 231us (3413us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302586.646987:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 189 00000100:00100000:1.0:1789302586.647648:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350164096 00000100:00100000:1.0:1789302586.647662:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350164096 00000100:00100000:1.0:1789302586.647696:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 190 00000100:00100000:1.0:1789302586.647703:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2347:x1876219350164096:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302586.647744:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2347:x1876219350164096:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 42us (1475us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302586.647751:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 190 00000100:00100000:1.0:1789302586.894346:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371040384:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302586.894470:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302586.894544:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371040384 00000100:00100000:2.0:1789302586.894556:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371040384 00000100:00100000:2.0:1789302586.894633:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 100 00000100:00100000:2.0:1789302586.894646:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371040384:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302586.894694:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 52756480 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302586.895991:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371040384:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1340us (1514us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302586.896022:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 100 00000100:00100000:1.0:1789302586.896136:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371040384:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302587.149882:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302588.045637:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302588.057281:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302588.057297:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302588.059083:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 11 count 1 00000040:00080000:1.0:1789302588.059099:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302590.095672:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302590.207425:0:15231:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: oleg311-client.virtnet: executing get_client_uuid /mnt/lustre2 00000100:00100000:2.0:1789302591.145726:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371040640:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302591.145764:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dd200/0x64fe43095184bcc2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789302591.151259:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302591.160242:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184bcc9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.160260:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184bcc9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.160268:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184bcc9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.160274:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302591.160279:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184bcc9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.160287:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184bcc9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.160304:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4f27b7ce (ffff960f02dfdd00) 00010000:00010000:2.0:1789302591.160308:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302591.160313:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184bcc9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.160320:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184bcc9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302591.172366:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302591.194898:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350164992 00000100:00100000:1.0:1789302591.194912:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350164992 00000100:00100000:1.0:1789302591.195021:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 532 00000100:00100000:1.0:1789302591.195033:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+54:17666:x1876219350164992:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302591.195051:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302591.195071:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184bcde lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb77 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195137:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195161:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195173:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195188:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195195:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195201:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195290:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bcf3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195298:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bcf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195305:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bcf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302591.195330:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x21:0x0]: returning size 16384 00000004:00200000:1.0:1789302591.195334:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302591.195352:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bcf3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195365:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195382:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195389:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195396:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302591.195401:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195408:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195420:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168400) 00010000:00010000:1.0:1789302591.195423:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302591.195428:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195435:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bce5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195448:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195454:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195460:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195464:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302591.195481:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195487:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195495:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4f27b7ce (ffff960f421e9b00) 00010000:00010000:1.0:1789302591.195498:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302591.195528:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195534:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bcec lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4f27b7ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195565:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184bcde lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb77 expref: 56 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195578:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99600/0x64fe43095184bcf3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb77 expref: 55 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.195586:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff960f421e9200) 00010000:00010000:1.0:1789302591.195588:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302591.195590:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99600, rc 0) 00000100:00100000:1.0:1789302591.195655:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+55:17666:x1876219350164992:12345-192.168.203.11@tcp:101:bash.0 Request processed in 625us (2824us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302591.195663:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 532 00000100:00100000:1.0:1789302591.200957:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350165120 00000100:00100000:1.0:1789302591.200967:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350165120 00000100:00100000:1.0:1789302591.201018:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00100000:1.0:1789302591.201030:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+55:17666:x1876219350165120:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302591.201058:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302591.201081:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bcfa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb7e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201146:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201161:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201171:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201185:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201191:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201197:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201253:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201260:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201268:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201274:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302591.201284:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201294:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201306:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff960f421e9200) 00010000:00010000:1.0:1789302591.201309:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302591.201315:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201321:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd01 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201333:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201339:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201345:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201350:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302591.201353:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201360:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201367:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x3953761c (ffff960f421e9600) 00010000:00010000:1.0:1789302591.201370:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302591.201380:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201386:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bd08 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201420:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bcfa lrc: 1/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb7e expref: 56 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201431:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff960f421e9200) 00010000:00010000:1.0:1789302591.201434:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302591.201438:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184bcfa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb7e expref: 56 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.201449:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99e00, rc 0) 00000100:00100000:1.0:1789302591.201902:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+55:17666:x1876219350165120:12345-192.168.203.11@tcp:101:bash.0 Request processed in 873us (1107us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302591.201911:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00100000:2.0:1789302591.219424:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350165248 00000100:00100000:2.0:1789302591.219434:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350165248 00000100:00100000:2.0:1789302591.219492:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 534 00000100:00100000:2.0:1789302591.219544:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:17667:x1876219350165248:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302591.219563:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302591.219594:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bd0f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fb85 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.219663:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.219677:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.219687:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302591.219723:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302591.219727:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302591.219743:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.219764:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bd0f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 21 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.219780:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 20 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.219787:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168400) 00010000:00010000:2.0:1789302591.219802:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302591.219806:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa800, rc 0) 00000100:00100000:2.0:1789302591.219854:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:17667:x1876219350165248:12345-192.168.203.11@tcp:101:rm.0 Request processed in 354us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302591.219861:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 534 00000100:00100000:1.0:1789302591.225758:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302591.255831:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302591.271141:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302591.303299:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302591.308076:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302591.317478:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 540 00000100:00100000:1.0:1789302591.317557:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+21:17668:x1876219350166144:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302591.317580:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302591.317604:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bd8d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fbaf expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.317746:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.317766:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.317777:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.317793:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd9b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.317800:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd9b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.317808:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd9b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302591.318492:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xc:0x0], index 1: 0 00010000:00010000:1.0:1789302591.318711:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bda2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318722:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bda2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318731:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bda2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318765:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318783:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318791:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318797:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302591.318802:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318810:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318823:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff960f421e9200) 00010000:00010000:1.0:1789302591.318826:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302591.318832:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318838:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184bd94 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302591.318852:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000068b2fa7a reply state 00000000e0a74b8e transno 4294967686 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184bd9b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x21:0x0].0x3953761c bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302591.318861:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302591.318892:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184bd8d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbaf expref: 24 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318908:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bda2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbaf expref: 23 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.318928:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff960f421e9b00) 00010000:00010000:1.0:1789302591.318931:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302591.318933:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc98800, rc 0) 00000100:00100000:1.0:1789302591.319015:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:17668:x1876219350166144:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1525us (4967us total) trans 4294967686 rc 0/0 00000100:00100000:1.0:1789302591.319025:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 540 00000100:00100000:2.0:1789302591.329757:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350166272 00000100:00100000:2.0:1789302591.329778:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350166272 00000100:00100000:2.0:1789302591.330293:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 135 00000100:00100000:2.0:1789302591.330311:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350166272:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302591.330336:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302591.330365:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fbb6 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302591.330404:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xc:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302591.330430:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=428s v=(0 0) 00010000:00010000:2.0:1789302591.330435:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302591.330447:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 7 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.330468:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 7 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.330485:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fbe00, rc 0) 00000100:00100000:2.0:1789302591.331670:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350166272:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1358us (3607us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302591.331681:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 135 00000100:00100000:1.0:1789302591.341060:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.341166:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350166400 00000100:00100000:2.0:1789302591.341270:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350166400 00000100:00100000:2.0:1789302591.341345:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 136 00000100:00100000:2.0:1789302591.341357:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:17668:x1876219350166400:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302591.341383:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302591.341410:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fbbd expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.341456:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 8 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.341467:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302591.341473:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302591.341479:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 4/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 8 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.341754:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 8 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.341772:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 8 pid: 6695 timeout: 528 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302591.341790:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef79c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371040768:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302591.341959:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302591.348197:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef79c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371040768:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302591.348239:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 8 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.348260:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa000, rc 0) 00000100:00100000:1.0:1789302591.348342:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:17668:x1876219350166400:12345-192.168.203.11@tcp:101:cp.0 Request processed in 6986us (7290us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302591.348351:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 136 00000100:00100000:2.0:1789302591.348969:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350166528 00000100:00100000:2.0:1789302591.348981:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350166528 00010000:00010000:2.0:1789302591.349042:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 8 pid: 6695 timeout: 528 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302591.349114:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 169 00000100:00100000:2.0:1789302591.349126:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350166528:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302591.349144:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302591.349176:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 8 pid: 6695 timeout: 528 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.349325:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 8 pid: 6695 timeout: 528 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.349356:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000008525adaa/0x64fe43095184bdb0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbb6 expref: 8 pid: 6695 timeout: 528 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.349376:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302591.349381:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302591.349390:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 7 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.349401:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 7 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302591.349450:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 7 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302591.349468:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371040896:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302591.349730:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ee40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302591.356702:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.448269:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:2.0:1789302591.449708:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302591.459188:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.463861:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302591.463999:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350167296 00000100:00100000:1.0:1789302591.464014:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350167296 00000100:00100000:1.0:1789302591.464104:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 138 00000100:00100000:1.0:1789302591.464117:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350167296:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302591.464144:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302591.464295:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000790d8d65/0x64fe43095184bdef lrc: 2/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fbd9 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302591.464400:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371041152:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302591.464569:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302591.468287:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371041152:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302591.468343:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000790d8d65/0x64fe43095184bdef lrc: 1/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbd9 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.468375:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000790d8d65/0x64fe43095184bdef lrc: 0/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbd9 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302591.468390:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99e00, rc 0) 00000100:00100000:1.0:1789302591.468474:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350167296:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4359us (4635us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302591.468484:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 138 00000100:00100000:1.0:1789302591.476602:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350167552 00000100:00100000:1.0:1789302591.476616:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350167552 00000100:00100000:1.0:1789302591.476683:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 543 00000100:00100000:1.0:1789302591.476696:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:17671:x1876219350167552:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:1.0:1789302591.477078:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].12, 136 off9824 00000100:00100000:1.0:1789302591.477192:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:17671:x1876219350167552:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 498us (764us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302591.477201:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 543 00000100:00100000:2.0:1789302591.501176:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371041280:0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.501289:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302591.501339:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371041792:0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.501371:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302591.501401:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302591.501407:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371041664:0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.501434:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302591.501468:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371041280 00000100:00100000:2.0:1789302591.501478:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371041280 00000100:00100000:2.0:1789302591.501586:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:2.0:1789302591.501598:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371041280:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.501661:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371041280:12345-0@lo:400:kworker.0 Request processed in 64us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302591.501668:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:2.0:1789302591.501705:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371041792 00000100:00100000:2.0:1789302591.501709:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371041792 00000100:00100000:2.0:1789302591.501749:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:2.0:1789302591.501756:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371041792:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.501791:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371041792:12345-0@lo:400:kworker.0 Request processed in 36us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302591.501797:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00080000:2.0:1789302591.501833:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302591.501867:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371041280:0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.501880:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371041792:0@lo:400:kworker.0 00000100:00100000:1.0:1789302591.502796:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371041408:0@lo:400:kworker.0 00000100:00100000:1.0:1789302591.502849:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302591.502897:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371041920:0@lo:400:kworker.0 00000100:00100000:1.0:1789302591.502926:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302591.502954:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302591.502960:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371041536:0@lo:400:kworker.0 00000100:00100000:1.0:1789302591.502983:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302591.503022:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371041664 00000100:00100000:1.0:1789302591.503029:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371041664 00000100:00100000:1.0:1789302591.503465:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371041920 00000100:00100000:1.0:1789302591.503470:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371041920 00000100:00080000:2.0:1789302591.504371:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302591.504376:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302591.504379:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302591.504419:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371041536:0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.504432:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371041408:0@lo:400:kworker.0 00000100:00100000:1.0:1789302591.519935:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:1.0:1789302591.519974:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371041920:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302591.523798:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371041920:12345-0@lo:400:kworker.0 Request processed in 3832us (20855us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302591.523819:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:1.0:1789302591.523919:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371041920:0@lo:400:kworker.0 00000100:00100000:1.0:1789302591.524039:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371041664:0@lo:400:kworker.0 00000100:00100000:2.0:1789302591.756362:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.756449:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.756472:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.756565:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.756594:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.756609:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.759667:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302591.759717:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350167680 00000100:00100000:2.0:1789302591.759729:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350167680 00000100:00100000:2.0:1789302591.759787:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:2.0:1789302591.759798:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350167680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302591.759878:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350167680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 81us (3311us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302591.759886:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:2.0:1789302591.759925:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350168448 00000100:00100000:2.0:1789302591.759929:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350168448 00000100:00100000:2.0:1789302591.759957:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 196 00000100:00100000:2.0:1789302591.759963:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2346:x1876219350168448:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302591.759998:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2346:x1876219350168448:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302591.760004:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 196 00000100:00080000:2.0:1789302591.760022:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302592.015563:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371042048:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302592.015686:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302592.023409:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371042048:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302592.141144:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302592.157464:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302592.190396:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371042304:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302592.190429:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 3/1,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.204093:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302592.204133:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 10 pid: 13439 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.204150:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 10 pid: 13439 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.204179:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006cddb364/0x64fe43095184bdb7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xc:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbbd expref: 10 pid: 13439 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.204196:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302592.204201:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302592.204207:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.204219:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 4/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.204263:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302592.204344:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350169344:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 274us (2110us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302592.204352:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 170 00010000:00010000:1.0:1789302592.204410:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204421:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204464:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204473:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204481:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204488:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302592.204493:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204533:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204546:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.204555:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038ce5aa0/0x64fe43095184be0b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302592.204613:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350168960:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 18318us (19351us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302592.204623:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 139 00000100:00100000:1.0:1789302592.217997:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302592.246782:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302592.246910:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350169728 00000100:00100000:3.0:1789302592.246926:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350169728 00000100:00100000:3.0:1789302592.247002:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 140 00000100:00100000:3.0:1789302592.247016:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350169728:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302592.247041:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302592.247070:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002905c0b1/0x64fe43095184be12 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fbe7 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302592.247111:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xc:0x0] initial LVB size: 1516, mtime: 0x6aa6973f, atime: 0x0, ctime: 0x6aa6973f, blocks: 0x1 00010000:00010000:3.0:1789302592.247138:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=429s v=(0 0) 00010000:00010000:3.0:1789302592.247142:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302592.247152:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002905c0b1/0x64fe43095184be12 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbe7 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.247172:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002905c0b1/0x64fe43095184be12 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbe7 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.247188:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f121dde00, rc 0) 00000100:00100000:3.0:1789302592.247263:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350169728:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 251us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302592.247271:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 140 00000100:00100000:2.0:1789302592.251414:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302592.251535:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350169984 00000100:00100000:3.0:1789302592.251551:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350169984 00000100:00100000:3.0:1789302592.251638:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 547 00000100:00100000:3.0:1789302592.251653:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:17681:x1876219350169984:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302592.251835:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:17681:x1876219350169984:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 186us (433us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302592.251843:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 547 00000100:00100000:3.0:1789302592.255131:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302592.255293:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350170112 00000100:00100000:1.0:1789302592.255306:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350170112 00002000:00100000:1.0:1789302592.255366:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xc:0x0] (0->4095) req@ffff960f3ad46d80 x1876219350170112/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:168/0 lens 488/0 e 0 to 0 dl 1789302603 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302592.255378:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad46d80 with x1876219350170112 ext(0->4095) 00002000:00010000:1.0:1789302592.255384:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ad46d80 00000100:00100000:1.0:1789302592.255419:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 51 00000100:00100000:1.0:1789302592.255431:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350170112:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302592.255719:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350170112:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 289us (604us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302592.255727:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 51 00002000:00100000:1.0:1789302592.255735:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xc:0x0] (0->4095) req@ffff960f3ad46d80 x1876219350170112/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:168/0 lens 488/4536 e 0 to 0 dl 1789302603 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302592.255744:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad46d80 with x1876219350170112 ext(0->4095) 00002000:00010000:1.0:1789302592.255748:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ad46d80 00000100:00100000:3.0:1789302592.261356:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302592.261941:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350170240 00000100:00100000:1.0:1789302592.261956:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350170240 00000100:00100000:1.0:1789302592.262037:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 548 00000100:00100000:1.0:1789302592.262049:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:17681:x1876219350170240:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302592.262078:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302592.262106:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184be19 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fbee expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.262165:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.262202:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.263236:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.263492:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184be19 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.264735:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.264751:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff960f421e9b00) 00010000:00010000:1.0:1789302592.264755:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302592.264759:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc98600, rc 0) 00000100:00100000:1.0:1789302592.264850:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:17681:x1876219350170240:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2801us (3509us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302592.264859:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 548 00000100:00100000:1.0:1789302592.273783:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302592.283563:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302592.283668:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350170496 00000100:00100000:2.0:1789302592.283680:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350170496 00000100:00100000:2.0:1789302592.283744:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 141 00000100:00100000:2.0:1789302592.283756:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350170496:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302592.283816:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283839:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302592.283844:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xc:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302592.283851:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283868:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283925:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283934:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283942:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283950:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302592.283955:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283965:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.283983:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.284275:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009bac0f50/0x64fe43095184be27 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xc:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302592.284485:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350170496:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 730us (937us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302592.284494:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 141 00000100:00100000:1.0:1789302592.305954:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350170752 00000100:00100000:1.0:1789302592.305969:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350170752 00000100:00100000:1.0:1789302592.306056:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 550 00000100:00100000:1.0:1789302592.306070:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:17681:x1876219350170752:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1789302592.306280:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.306303:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.306331:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 4/0,1 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.306382:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 26 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302592.306400:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 26 pid: 5844 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302592.306418:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371042432:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302592.308458:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03818b40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302592.320409:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371042432:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302592.320456:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302592.325761:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350170880 00000100:00100000:2.0:1789302592.325778:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350170880 00010000:00010000:2.0:1789302592.325836:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 27 pid: 5844 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302592.325885:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 171 00000100:00100000:2.0:1789302592.325897:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:2348:x1876219350170880:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302592.325918:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302592.325926:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 27 pid: 5844 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.325943:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 27 pid: 5844 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.325969:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184be20 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbee expref: 27 pid: 5844 timeout: 529 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.325990:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff960f421e9b00) 00010000:00010000:2.0:1789302592.325996:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,1 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.326006:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.326015:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302592.326047:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302592.326128:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:2348:x1876219350170880:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 232us (1264us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302592.326145:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 171 00000040:00080000:1.0:1789302592.328006:0:15333:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302592.328018:0:15333:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302592.328080:0:15333:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 13 count 2 00000040:00080000:1.0:1789302592.328086:0:15333:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302592.328999:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329020:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329317:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329326:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329335:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329342:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302592.329348:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329358:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329379:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff960f421e9b00) 00010000:00010000:3.0:1789302592.329382:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302592.329389:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 2/0,0 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302592.329396:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98e00/0x64fe43095184be2e lrc: 0/0,0 mode: --/PW res: [0x200000402:0xd:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302592.330777:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:17681:x1876219350170752:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 24697us (24921us total) trans 4294967689 rc 0/0 00000100:00100000:3.0:1789302592.330802:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 550 00000040:00080000:3.0:1789302592.330992:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302592.330998:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302592.331482:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:3.0:1789302592.331491:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302592.341481:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302592.342172:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350171136 00000100:00100000:2.0:1789302592.342262:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350171136 00000100:00100000:2.0:1789302592.342353:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 120 00000100:00100000:2.0:1789302592.342367:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:17681:x1876219350171136:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302592.343944:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:17681:x1876219350171136:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1571us (2470us total) trans 4294967690 rc 0/0 00000100:00100000:2.0:1789302592.343960:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 120 00000100:00100000:1.0:1789302592.525688:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371042560:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302592.525787:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302592.525879:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371042560 00000100:00100000:3.0:1789302592.525890:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371042560 00000100:00100000:3.0:1789302592.525956:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 103 00000100:00100000:3.0:1789302592.525968:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371042560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302592.526015:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 56164352 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302592.526092:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371042560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 126us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302592.526100:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 103 00000100:00100000:1.0:1789302592.526215:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371042560:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302593.165291:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302594.193669:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302594.254883:0:15384:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302594.254891:0:15384:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302594.254928:0:15384:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 13 count 2 00000040:00080000:1.0:1789302594.254933:0:15384:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302594.315632:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350171392 00000100:00100000:2.0:1789302594.315644:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350171392 00000100:00100000:2.0:1789302594.315722:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 551 00000100:00100000:2.0:1789302594.315733:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:17692:x1876219350171392:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302594.315754:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302594.315774:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184be35 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fbf5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302594.315877:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184be3c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302594.315893:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184be3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302594.315904:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184be3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302594.315915:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xd:0x0]: returning size 1516 00010000:00010000:2.0:1789302594.316054:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184be35 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x21:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbf5 expref: 60 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302594.316089:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184be3c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbf5 expref: 59 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302594.316099:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff960f421e9b00) 00010000:00010000:2.0:1789302594.316102:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302594.316105:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa000, rc 0) 00000100:00100000:2.0:1789302594.316166:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:17692:x1876219350171392:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 435us (1220us total) trans 4294967691 rc 0/0 00000100:00100000:2.0:1789302594.316175:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 551 00000100:00100000:3.0:1789302594.326407:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350171520 00000100:00100000:3.0:1789302594.326412:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350171520 00000100:00100000:3.0:1789302594.326540:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 552 00000100:00100000:3.0:1789302594.326549:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+58:17692:x1876219350171520:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:3.0:1789302594.326604:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 3/1,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326619:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326629:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326665:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326672:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326678:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326685:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f121ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302594.326689:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326696:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326706:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff960f421e9b00) 00010000:00010000:3.0:1789302594.326708:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302594.326712:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 2/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302594.326719:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be4a lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302594.328558:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+58:17692:x1876219350171520:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 2000us (2613us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302594.328572:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 552 00000100:00100000:1.0:1789302594.346027:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302594.346242:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350171904 00000100:00100000:2.0:1789302594.346252:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350171904 00000100:00100000:2.0:1789302594.346312:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 121 00000100:00100000:2.0:1789302594.346321:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+58:17692:x1876219350171904:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:2.0:1789302594.346592:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:17692:x1876219350171904:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 271us (573us total) trans 4294967692 rc 0/0 00000100:00100000:2.0:1789302594.346599:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 121 00010000:00010000:3.0:1789302595.213189:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302595.693182:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967687 is committed 00000001:00080000:1.0:1789302595.693202:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967685 is committed 00000001:00080000:1.0:1789302595.693230:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967692 is committed 00000001:00080000:1.0:1789302595.693238:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967690 is committed 00010000:00010000:1.0:1789302596.237819:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302596.620988:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371042688:0@lo:400:kworker.0 00000100:00100000:2.0:1789302596.621192:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302596.621438:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371042688 00000100:00100000:3.0:1789302596.621445:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371042688 00000100:00100000:1.0:1789302596.621700:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371043072:0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.621735:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302596.621783:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371042816:0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.621809:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302596.621826:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302596.621832:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371042944:0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.621851:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302596.621876:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371043072 00000100:00100000:1.0:1789302596.621880:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371043072 00000100:00100000:1.0:1789302596.621917:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:1.0:1789302596.621926:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371043072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.621970:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371043072:12345-0@lo:400:kworker.0 Request processed in 46us (234us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302596.621976:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:1.0:1789302596.622000:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371042816 00000100:00100000:1.0:1789302596.622003:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371042816 00000100:00100000:1.0:1789302596.622025:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371042944 00000100:00100000:1.0:1789302596.622027:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371042944 00000100:00100000:1.0:1789302596.622045:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:1.0:1789302596.622051:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371042816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.622094:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371042816:12345-0@lo:400:kworker.0 Request processed in 43us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302596.622100:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:1.0:1789302596.622110:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:1.0:1789302596.622135:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371042944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.622161:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371042944:12345-0@lo:400:kworker.0 Request processed in 40us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302596.622167:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00080000:1.0:1789302596.622184:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302596.622188:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302596.622230:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371043072:0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.622257:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371042944:0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.622265:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371042816:0@lo:400:kworker.0 00000100:00100000:3.0:1789302596.622486:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:3.0:1789302596.622495:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371042688:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.623381:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302596.623393:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371043328:0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.623430:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302596.623484:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302596.623492:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371043200:0@lo:400:kworker.0 00000100:00100000:1.0:1789302596.623534:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1789302596.624248:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302596.624275:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371043328:0@lo:400:kworker.0 00000100:00080000:1.0:1789302596.624900:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302596.624922:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371043200:0@lo:400:kworker.0 00000100:00100000:3.0:1789302596.625302:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371042688:12345-0@lo:400:kworker.0 Request processed in 2808us (4109us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302596.625311:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:2.0:1789302596.626950:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371042688:0@lo:400:kworker.0 00000100:00100000:3.0:1789302596.880366:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302596.880419:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302596.880439:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302596.880462:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302596.880496:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302596.880539:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302596.880564:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302596.880592:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172416 00000100:00100000:3.0:1789302596.880601:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172416 00000100:00100000:3.0:1789302596.880652:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172288 00000100:00100000:3.0:1789302596.880656:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172288 00000100:00100000:3.0:1789302596.880668:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172800 00000100:00100000:3.0:1789302596.880671:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172800 00000100:00100000:3.0:1789302596.880682:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172672 00000100:00100000:3.0:1789302596.880684:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172672 00000100:00100000:3.0:1789302596.880700:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 199 00000100:00100000:3.0:1789302596.880709:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350172416:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302596.880774:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350172416:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 66us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302596.880782:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 199 00000100:00100000:3.0:1789302596.880801:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 200 00000100:00100000:3.0:1789302596.880807:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350172288:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302596.880840:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350172288:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (379us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302596.880846:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 200 00000100:00100000:3.0:1789302596.880858:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 201 00000100:00100000:3.0:1789302596.880865:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350172800:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302596.880885:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350172800:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302596.880891:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 201 00000100:00100000:3.0:1789302596.880903:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 202 00000100:00100000:3.0:1789302596.880910:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350172672:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302596.880938:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2348:x1876219350172672:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302596.880945:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 202 00000100:00100000:1.0:1789302596.881132:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172160 00000100:00100000:1.0:1789302596.881137:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172160 00000100:00100000:1.0:1789302596.881177:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172544 00000100:00100000:1.0:1789302596.881181:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172544 00000100:00100000:1.0:1789302596.881210:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:1.0:1789302596.881218:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:2348:x1876219350172160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302596.881266:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:2348:x1876219350172160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 48us (910us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302596.881273:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:1.0:1789302596.881289:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 244 00000100:00100000:1.0:1789302596.881296:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350172544:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302596.881322:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350172544:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (827us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302596.881329:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 244 00000100:00080000:1.0:1789302596.881355:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302596.881366:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302596.881368:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302596.881900:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172032 00000100:00100000:2.0:1789302596.881906:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172032 00000100:00100000:2.0:1789302596.881939:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:2.0:1789302596.881946:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350172032:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302596.881981:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350172032:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (1561us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302596.881997:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:3.0:1789302597.134693:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302597.134714:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371043456:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302597.134802:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302597.134866:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371043456 00000100:00100000:3.0:1789302597.134874:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371043456 00000100:00100000:3.0:1789302597.134917:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 104 00000100:00100000:3.0:1789302597.134926:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371043456:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302597.134960:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 54460416 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302597.135006:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371043456:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302597.135013:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 104 00000100:00100000:3.0:1789302597.135084:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371043456:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302597.264320:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302597.646003:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302597.649268:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371043584:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302597.649389:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302597.649906:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371043584 00000100:00100000:3.0:1789302597.649920:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371043584 00000100:00100000:3.0:1789302597.649996:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 105 00000100:00100000:3.0:1789302597.650009:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371043584:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302597.650071:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 56164352 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302597.650301:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371043584:12345-0@lo:13:osp-pre-0-0.0 Request processed in 294us (912us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302597.650310:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 105 00000100:00100000:2.0:1789302597.650876:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371043584:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302598.286778:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302599.001148:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350172928 00000100:00100000:1.0:1789302599.001166:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350172928 00000100:00080000:1.0:1789302599.001366:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302599.006024:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302599.006125:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350173184 00000100:00100000:3.0:1789302599.006147:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350173184 00000100:00100000:3.0:1789302599.006220:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 122 00000100:00100000:3.0:1789302599.006235:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:17151:x1876219350173184:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302599.006894:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:17151:x1876219350173184:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 660us (883us total) trans 4294967693 rc 0/0 00000100:00100000:3.0:1789302599.006904:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 122 00010000:00010000:3.0:1789302599.308901:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302600.332765:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1789302600.701080:0:15584:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 9a: Multiple remote agents ============ 08:29:58 (1789302598) 00000001:00080000:1.0:1789302600.789306:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967693 is committed 00010000:00010000:3.0:1789302601.357194:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302601.741911:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371043840:0@lo:400:kworker.0 00000100:00100000:2.0:1789302601.742004:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302601.742069:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371044352:0@lo:400:kworker.0 00000100:00100000:2.0:1789302601.742098:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302601.742135:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302601.742141:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371043968:0@lo:400:kworker.0 00000100:00100000:2.0:1789302601.742170:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302601.742209:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371043840 00000100:00100000:2.0:1789302601.742218:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371043840 00000100:00100000:2.0:1789302601.742271:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371043968 00000100:00100000:2.0:1789302601.742274:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371043968 00000100:00100000:2.0:1789302601.742304:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1789302601.742313:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371043840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302601.742374:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371043840:12345-0@lo:400:kworker.0 Request processed in 62us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302601.742383:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:2.0:1789302601.742404:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1789302601.742410:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371043968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302601.742451:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371043968:12345-0@lo:400:kworker.0 Request processed in 41us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302601.742465:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1789302601.742486:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371044352 00000100:00100000:2.0:1789302601.742489:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371044352 00000100:00100000:2.0:1789302601.742559:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:1.0:1789302601.742709:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371043712:0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.742758:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302601.742787:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371044224:0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.742810:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302601.742830:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789302601.742836:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371044096:0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.742858:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302601.742886:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371043712 00000100:00100000:1.0:1789302601.742897:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371043712 00000100:00100000:1.0:1789302601.742930:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:1.0:1789302601.742937:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371043712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.742973:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371043712:12345-0@lo:400:kworker.0 Request processed in 36us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302601.742980:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:1.0:1789302601.743008:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371044096 00000100:00100000:1.0:1789302601.743012:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371044096 00000100:00100000:1.0:1789302601.743036:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789302601.743042:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371044096:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.743071:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371044096:12345-0@lo:400:kworker.0 Request processed in 29us (212us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302601.743077:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789302601.743121:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371044096:0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.743130:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371043712:0@lo:400:kworker.0 00000100:00100000:3.0:1789302601.743274:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371044224 00000100:00100000:3.0:1789302601.743279:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371044224 00000100:00100000:3.0:1789302601.743304:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:3.0:1789302601.743310:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371044224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302601.743357:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371044224:12345-0@lo:400:kworker.0 Request processed in 47us (546us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302601.743363:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00080000:3.0:1789302601.743390:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302601.743395:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302601.743398:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302601.743455:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371043968:0@lo:400:kworker.0 00000100:00100000:3.0:1789302601.743468:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371043840:0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.743808:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371044224:0@lo:400:kworker.0 00000100:00100000:2.0:1789302601.745337:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371044352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302601.747224:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371044352:12345-0@lo:400:kworker.0 Request processed in 4649us (5115us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302601.747241:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1789302601.749340:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371044352:0@lo:400:kworker.0 00000100:00100000:1.0:1789302601.998472:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302601.998641:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302601.999401:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302601.999446:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302602.001823:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302602.001856:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302602.001970:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350173568 00000100:00100000:3.0:1789302602.001979:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350173568 00000100:00100000:3.0:1789302602.002044:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350174080 00000100:00100000:3.0:1789302602.002047:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350174080 00000100:00100000:3.0:1789302602.002075:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 205 00000100:00100000:3.0:1789302602.002085:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2346:x1876219350173568:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302602.002168:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2346:x1876219350173568:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 85us (3528us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302602.002175:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 205 00000100:00100000:3.0:1789302602.002193:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 206 00000100:00100000:3.0:1789302602.002687:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2346:x1876219350174080:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302602.002804:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2346:x1876219350174080:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 603us (3355us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302602.002812:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 206 00000100:00100000:3.0:1789302602.002849:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350173440 00000100:00100000:3.0:1789302602.002855:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350173440 00000100:00100000:3.0:1789302602.002893:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:3.0:1789302602.002902:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:2346:x1876219350173440:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302602.002935:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:2346:x1876219350173440:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (1124us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302602.002958:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00080000:3.0:1789302602.002986:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302602.002990:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302602.002993:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302602.003914:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350173696 00000100:00100000:1.0:1789302602.003924:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350173696 00000100:00100000:1.0:1789302602.003973:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 207 00000100:00100000:1.0:1789302602.003981:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350173696:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302602.004032:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350173696:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 52us (2176us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302602.004039:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 207 00000100:00080000:3.0:1789302602.004279:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302602.004817:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302602.004939:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350173952 00000100:00100000:3.0:1789302602.004947:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350173952 00000100:00100000:3.0:1789302602.005002:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00100000:3.0:1789302602.005010:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2346:x1876219350173952:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302602.005058:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2346:x1876219350173952:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 49us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302602.005065:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00080000:1.0:1789302602.005361:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:02000400:3.0:1789302602.159231:0:15635:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-hsm test_9a Need 3 or more clients, have 1 00000100:00100000:1.0:1789302602.253372:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371044480:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302602.253477:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302602.253834:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371044480 00000100:00100000:3.0:1789302602.253848:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371044480 00000100:00100000:3.0:1789302602.253914:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 106 00000100:00100000:3.0:1789302602.253924:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371044480:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302602.253958:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 54460416 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302602.254049:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371044480:12345-0@lo:13:osp-pre-1-0.0 Request processed in 127us (573us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302602.254057:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 106 00010000:00010000:1.0:1789302602.380679:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302602.770453:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371044608:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302602.770609:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302602.771043:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371044608 00000100:00100000:3.0:1789302602.771054:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371044608 00000100:00100000:3.0:1789302602.771121:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 107 00000100:00100000:3.0:1789302602.771133:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371044608:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302602.771182:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 56164352 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302602.771271:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371044608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 140us (661us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302602.771279:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 107 00010000:00010000:3.0:1789302603.407782:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1789302603.858231:0:15683:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10a: Archive a file =================== 08:30:02 (1789302602) 00000100:00100000:3.0:1789302604.247672:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:1.0:1789302604.255206:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302604.264658:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:1.0:1789302604.265678:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302604.265978:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.266170:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350174592 00000100:00100000:3.0:1789302604.266177:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350174592 00000100:00100000:2.0:1789302604.266329:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302604.266351:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350174720 00000100:00100000:2.0:1789302604.266355:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350174720 00000100:00100000:2.0:1789302604.266379:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 556 00000100:00100000:2.0:1789302604.266386:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:18587:x1876219350174720:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302604.266463:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:18587:x1876219350174720:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 76us (133us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302604.266470:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 556 00000100:00100000:3.0:1789302604.267191:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 18 00000100:00100000:3.0:1789302604.267202:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2345:x1876219350174592:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302604.267215:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302604.267238:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184be66 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbfc expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.267256:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302604.267260:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302604.267294:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:2345:x1876219350174592:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 95us (1317us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.267311:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 18 00000100:00100000:2.0:1789302604.278980:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.279152:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350174976 00000100:00100000:3.0:1789302604.279162:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350174976 00000100:00100000:3.0:1789302604.279228:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 557 00000100:00100000:3.0:1789302604.279239:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:18588:x1876219350174976:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:3.0:1789302604.279296:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302604.279338:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184bcd7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb70 expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.279376:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302604.279382:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302604.279628:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.279643:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 22 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.279653:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.279688:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 22 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.280092:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 22 pid: 5843 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302604.280111:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371044736:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302604.280221:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4b40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302604.284136:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350175104 00000100:00100000:1.0:1789302604.284152:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350175104 00010000:00010000:1.0:1789302604.284204:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 23 pid: 5843 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302604.284434:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371044736:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302604.284470:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302604.285451:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 172 00000100:00100000:1.0:1789302604.285478:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:2347:x1876219350175104:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302604.285497:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302604.285520:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 23 pid: 5843 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.285537:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 23 pid: 5843 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.285584:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bd16 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb85 expref: 23 pid: 5843 timeout: 541 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.285602:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168400) 00010000:00010000:1.0:1789302604.285608:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.285618:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.285625:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.285660:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302604.285710:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.285723:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.285773:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.285786:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.285796:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302604.285820:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350175104:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 350us (1803us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302604.285830:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 172 00000004:00200000:3.0:1789302604.289896:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00010000:00010000:3.0:1789302604.289925:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.289938:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.289947:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.289953:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f121ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.289959:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.289969:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.289989:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302604.289993:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.289998:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.290024:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184be89 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302604.290439:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c5c96139 reply state 000000001b4c038a transno 4294967695 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302604.290451:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302604.290667:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:18588:x1876219350174976:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 11423us (11689us total) trans 4294967695 rc 0/0 00000100:00100000:3.0:1789302604.290679:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 557 00010000:00010000:2.0:1789302604.294688:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.294707:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.294715:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.294721:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6f7e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302604.294752:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.294761:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.294787:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff960f14168600) 00010000:00010000:2.0:1789302604.294791:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.294797:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.294803:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be90 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302604.308000:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.308090:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350175232 00000100:00100000:3.0:1789302604.308100:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350175232 00000100:00100000:3.0:1789302604.308159:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 558 00000100:00100000:3.0:1789302604.308169:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:18589:x1876219350175232:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302604.308194:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.308218:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184be97 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc0a expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.308286:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be9e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.308313:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be9e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.308325:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be9e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302604.308363:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302604.308367:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302604.308405:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be9e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.308442:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184be97 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc0a expref: 58 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.308463:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be9e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc0a expref: 57 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.308471:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302604.308474:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302604.308478:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6f7e00, rc 0) 00000100:00100000:3.0:1789302604.308610:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:18589:x1876219350175232:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 442us (619us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.308618:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 558 00000100:00100000:3.0:1789302604.312781:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350175360 00000100:00100000:3.0:1789302604.312791:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350175360 00000100:00100000:3.0:1789302604.312850:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 559 00000100:00100000:3.0:1789302604.312860:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:2345:x1876219350175360:12345-192.168.203.11@tcp:41:mkdir.0 00000004:00000024:3.0:1789302604.312908:0:5843:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 2259943424 avail 2257846272 00000100:00100000:3.0:1789302604.312973:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:2345:x1876219350175360:12345-192.168.203.11@tcp:41:mkdir.0 Request processed in 115us (282us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.312981:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 559 00000100:00100000:1.0:1789302604.313725:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.313781:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350175616 00000100:00100000:3.0:1789302604.313788:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350175616 00000100:00100000:3.0:1789302604.313827:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 560 00000100:00100000:3.0:1789302604.313835:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:18589:x1876219350175616:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:3.0:1789302604.313898:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302604.313924:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7e00/0x64fe43095184be9e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc0a expref: 57 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.313943:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302604.313953:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302604.314081:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:18589:x1876219350175616:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 247us (359us total) trans 0 rc -17/-17 00000100:00100000:3.0:1789302604.314088:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 560 00000100:00100000:2.0:1789302604.317199:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.317270:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350175744 00000100:00100000:3.0:1789302604.317280:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350175744 00000100:00100000:3.0:1789302604.317350:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 561 00000100:00100000:3.0:1789302604.317359:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+56:18589:x1876219350175744:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302604.317381:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.317406:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184bea5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc11 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.317478:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184beac lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.317491:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184beac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.317651:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184beac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302604.317689:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302604.317693:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302604.317728:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184beac lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.317754:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184bea5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc11 expref: 58 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.317771:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184beac lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc11 expref: 57 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.317780:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302604.317783:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302604.317787:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f121dca00, rc 0) 00000100:00100000:3.0:1789302604.317923:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcddc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:18589:x1876219350175744:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 565us (733us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.317930:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 561 00000100:00100000:3.0:1789302604.330219:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350175872 00000100:00100000:3.0:1789302604.330229:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350175872 00000100:00100000:3.0:1789302604.330287:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 562 00000100:00100000:3.0:1789302604.330309:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+57:18589:x1876219350175872:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302604.330330:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.330353:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184beb3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc18 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330430:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330445:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330475:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330544:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330553:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330561:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330648:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184bec8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330657:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184bec8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330664:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184bec8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302604.330694:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000004:00200000:3.0:1789302604.330698:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302604.330716:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184bec8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330728:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330735:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330750:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330757:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.330761:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330768:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330779:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302604.330783:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.330786:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330792:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184beba lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330803:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330809:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330815:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330820:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.330823:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330838:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330847:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff960f14168300) 00010000:00010000:3.0:1789302604.330849:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.330852:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330859:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bec1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330891:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6f7c00/0x64fe43095184beb3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc18 expref: 59 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330906:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184bec8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc18 expref: 58 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.330913:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302604.330915:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302604.330916:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d000, rc 0) 00000100:00100000:3.0:1789302604.330974:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+58:18589:x1876219350175872:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 678us (839us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.330981:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 562 00000100:00100000:1.0:1789302604.336208:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302604.339788:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184bed6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.339802:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184bed6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.339830:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184bed6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.339835:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.339839:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184bed6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.339846:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184bed6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.339860:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x43454b14 (ffff960f02dfd400) 00010000:00010000:3.0:1789302604.339863:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.339867:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184bed6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.339872:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184bed6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x22:0x0].0x43454b14 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302604.348372:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.348528:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350176256 00000100:00100000:3.0:1789302604.348539:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350176256 00000100:00100000:3.0:1789302604.348596:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 564 00000100:00100000:3.0:1789302604.348606:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22330c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+58:18590:x1876219350176256:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302604.348630:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.348652:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bedd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc1f expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348737:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348762:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348773:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348800:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348807:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348814:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348909:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c200/0x64fe43095184bef2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348917:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c200/0x64fe43095184bef2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348925:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c200/0x64fe43095184bef2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302604.348949:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000004:00200000:3.0:1789302604.348953:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302604.348969:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c200/0x64fe43095184bef2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348980:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.348995:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349002:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349007:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.349011:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349019:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349029:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302604.349032:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.349036:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349041:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bee4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349053:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349059:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349065:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349069:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2ca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.349076:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349082:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349090:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff960f14168300) 00010000:00010000:3.0:1789302604.349093:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.349095:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349102:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184beeb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349131:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bedd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc1f expref: 60 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349145:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c200/0x64fe43095184bef2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc1f expref: 59 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.349152:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:3.0:1789302604.349154:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789302604.349157:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c200, rc 0) 00000100:00100000:3.0:1789302604.349329:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22330c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:18590:x1876219350176256:12345-192.168.203.11@tcp:101:bash.0 Request processed in 725us (966us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.349336:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 564 00000100:00100000:3.0:1789302604.360783:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350176384 00000100:00100000:3.0:1789302604.360794:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350176384 00000100:00100000:3.0:1789302604.360850:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 565 00000100:00100000:3.0:1789302604.360860:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:18590:x1876219350176384:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302604.360895:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.360914:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bef9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc26 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.360983:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.360997:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361008:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361021:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361028:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361034:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361087:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361095:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361102:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361108:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2ca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.361119:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361128:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361139:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:3.0:1789302604.361142:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.361147:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361154:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf00 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361167:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361173:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361179:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361184:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.361188:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361193:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361200:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff960f14168300) 00010000:00010000:3.0:1789302604.361202:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.361212:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361218:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf07 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361252:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bef9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc26 expref: 60 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361261:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:3.0:1789302604.361264:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.361267:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bef9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc26 expref: 60 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.361276:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c000, rc 0) 00000100:00100000:3.0:1789302604.361391:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:18590:x1876219350176384:12345-192.168.203.11@tcp:101:bash.0 Request processed in 532us (695us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302604.361398:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 565 00000100:00100000:2.0:1789302604.391262:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350176512 00000100:00100000:2.0:1789302604.391271:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350176512 00000100:00100000:2.0:1789302604.391330:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 566 00000100:00100000:2.0:1789302604.391340:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:18591:x1876219350176512:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302604.391360:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302604.391381:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bf0e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc2d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.391455:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.391469:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.391480:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302604.391569:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302604.391573:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302604.391590:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.391614:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bf0e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 24 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.391631:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 23 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.391638:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:2.0:1789302604.391641:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302604.391645:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa800, rc 0) 00000100:00100000:2.0:1789302604.391696:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:18591:x1876219350176512:12345-192.168.203.11@tcp:101:rm.0 Request processed in 358us (602us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302604.391703:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 566 00000100:00100000:1.0:1789302604.401151:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.401265:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350176640 00000100:00100000:3.0:1789302604.401275:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350176640 00000100:00100000:3.0:1789302604.401337:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:3.0:1789302604.401348:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:18591:x1876219350176640:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302604.401368:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.401389:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bf1c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc34 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401466:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401499:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401547:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401564:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401572:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401580:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401668:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c600/0x64fe43095184bf31 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401678:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c600/0x64fe43095184bf31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401686:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c600/0x64fe43095184bf31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302604.401712:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000004:00200000:3.0:1789302604.401716:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302604.401731:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c600/0x64fe43095184bf31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401743:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401760:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401767:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401773:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.401777:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401785:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401796:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302604.401800:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.401804:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401811:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf23 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401825:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401831:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401837:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401843:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2ca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.401852:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401860:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401868:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd97ceb69 (ffff960f14168300) 00010000:00010000:3.0:1789302604.401871:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.401874:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401880:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf2a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd97ceb69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401912:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bf1c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc34 expref: 25 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401925:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c600/0x64fe43095184bf31 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc34 expref: 24 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.401932:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:3.0:1789302604.401934:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789302604.402082:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c600, rc 0) 00000100:00100000:3.0:1789302604.403061:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18591:x1876219350176640:12345-192.168.203.11@tcp:101:rm.0 Request processed in 1709us (1914us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.403070:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:2.0:1789302604.416016:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350176768 00000100:00100000:2.0:1789302604.416026:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350176768 00000100:00100000:2.0:1789302604.416085:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 568 00000100:00100000:2.0:1789302604.416095:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18591:x1876219350176768:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302604.416114:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302604.416135:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bf38 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc3b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416196:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416232:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416244:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416261:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416270:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416278:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416330:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416338:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416344:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416350:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302604.416354:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416361:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416373:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:2.0:1789302604.416383:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.416390:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416396:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf3f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416409:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416426:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416433:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416438:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fbe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302604.416440:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416447:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416454:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff960f11d36c00) 00010000:00010000:2.0:1789302604.416457:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.416460:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416465:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf46 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416499:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bf38 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc3b expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416557:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:2.0:1789302604.416560:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.416565:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bf38 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc3b expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.416576:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa000, rc 0) 00000100:00100000:2.0:1789302604.416695:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18591:x1876219350176768:12345-192.168.203.11@tcp:101:rm.0 Request processed in 601us (801us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302604.416702:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 568 00000100:00100000:3.0:1789302604.428996:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350176896 00000100:00100000:3.0:1789302604.429006:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350176896 00000100:00100000:3.0:1789302604.429063:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 569 00000100:00100000:3.0:1789302604.429073:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18591:x1876219350176896:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302604.429109:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.429135:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bf4d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc42 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429199:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429214:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429225:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429239:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429245:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429252:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429315:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429323:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429329:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429334:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2ca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.429344:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429351:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429363:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:3.0:1789302604.429366:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.429370:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429384:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf54 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429400:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429409:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429427:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429435:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.429438:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429445:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429457:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff960f14168300) 00010000:00010000:3.0:1789302604.429460:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.429469:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429481:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429645:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bf4d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc42 expref: 25 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429661:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:3.0:1789302604.429665:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.429669:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bf4d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc42 expref: 25 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.429680:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c000, rc 0) 00000100:00100000:3.0:1789302604.429780:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18591:x1876219350176896:12345-192.168.203.11@tcp:101:rm.0 Request processed in 708us (895us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302604.429788:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 569 00010000:00010000:1.0:1789302604.431199:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302604.444919:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302604.445118:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350177024 00000100:00100000:2.0:1789302604.445128:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350177024 00000100:00100000:2.0:1789302604.445190:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 570 00000100:00100000:2.0:1789302604.445200:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18592:x1876219350177024:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302604.445220:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302604.445239:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf62 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc49 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445301:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445316:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445327:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445341:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445367:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445374:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445437:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445445:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445451:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445457:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302604.445462:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445470:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445481:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:2.0:1789302604.445494:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.445634:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445643:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bf69 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445657:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445669:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445675:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445681:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302604.445684:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445691:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445700:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff960f11d36c00) 00010000:00010000:2.0:1789302604.445702:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.445705:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445710:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bf70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445742:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf62 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc49 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445753:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:2.0:1789302604.445755:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.445759:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bf62 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc49 expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.445780:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fbe00, rc 0) 00000100:00100000:2.0:1789302604.445876:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18592:x1876219350177024:12345-192.168.203.11@tcp:101:cp.0 Request processed in 678us (966us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302604.445884:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 570 00000100:00100000:2.0:1789302604.454694:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302604.454976:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350177152 00000100:00100000:1.0:1789302604.454988:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350177152 00000100:00100000:1.0:1789302604.455051:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 571 00000100:00100000:1.0:1789302604.455063:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18592:x1876219350177152:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302604.455102:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302604.455124:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bf77 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc50 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455188:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455203:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455215:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455229:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455237:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455244:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455295:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455303:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455310:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455316:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302604.455327:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455334:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455345:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:1.0:1789302604.455348:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302604.455352:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455359:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184bf7e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455374:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455380:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455386:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455391:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302604.455394:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455399:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455406:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff960f421e9700) 00010000:00010000:1.0:1789302604.455408:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302604.455426:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455432:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98400/0x64fe43095184bf85 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455464:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bf77 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc50 expref: 25 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455473:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:1.0:1789302604.455476:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302604.455479:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184bf77 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc50 expref: 25 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.455488:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99c00, rc 0) 00000100:00100000:1.0:1789302604.455649:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18592:x1876219350177152:12345-192.168.203.11@tcp:101:cp.0 Request processed in 587us (963us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302604.455656:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 571 00000100:00100000:1.0:1789302604.459868:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.459944:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350177408 00000100:00100000:3.0:1789302604.459956:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350177408 00000100:00100000:3.0:1789302604.460018:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 572 00000100:00100000:3.0:1789302604.460028:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18592:x1876219350177408:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302604.460048:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.460069:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bf8c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc57 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.460186:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.460220:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.460232:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.460249:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf9a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.460257:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf9a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.460265:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf9a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302604.460922:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xd:0x0], index 0: 0 00010000:00010000:3.0:1789302604.461087:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d800/0x64fe43095184bfa1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461100:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d800/0x64fe43095184bfa1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461110:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d800/0x64fe43095184bfa1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461144:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461152:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461161:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461174:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302604.461179:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461187:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461200:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:3.0:1789302604.461203:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302604.461208:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461214:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184bf93 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302604.461227:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003755f43b reply state 00000000ec9307a5 transno 4294967697 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184bf9a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302604.461236:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302604.461264:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184bf8c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc57 expref: 27 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461278:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d800/0x64fe43095184bfa1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc57 expref: 26 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.461285:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302604.461288:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302604.461290:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d800, rc 0) 00000100:00100000:3.0:1789302604.461360:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:18592:x1876219350177408:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1334us (1501us total) trans 4294967697 rc 0/0 00000100:00100000:3.0:1789302604.461375:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 572 00000100:00100000:1.0:1789302604.478305:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302604.478684:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350177536 00000100:00100000:2.0:1789302604.478694:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350177536 00000100:00100000:2.0:1789302604.478763:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 142 00000100:00100000:2.0:1789302604.478774:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+6:2347:x1876219350177536:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302604.478790:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302604.478808:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 2/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc5e expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302604.478835:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302604.478854:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=441s v=(0 0) 00010000:00010000:2.0:1789302604.478858:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302604.478867:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 7 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.478883:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 7 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.478897:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa400, rc 0) 00000100:00080000:1.0:1789302604.479124:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302604.479184:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350177536:12345-192.168.203.11@tcp:101:cp.0 Request processed in 411us (888us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302604.479208:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 142 00000100:00100000:2.0:1789302604.483145:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.483222:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350177664 00000100:00100000:3.0:1789302604.483233:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350177664 00000100:00100000:3.0:1789302604.483295:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 143 00000100:00100000:3.0:1789302604.483304:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:18592:x1876219350177664:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302604.483322:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.483339:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc65 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.483367:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 8 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.483376:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302604.483380:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302604.483386:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 4/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.483429:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 8 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.483443:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 8 pid: 6695 timeout: 541 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302604.483457:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db47b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371044864:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302604.484024:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302604.490858:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302604.490951:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350177792 00000100:00100000:2.0:1789302604.490961:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350177792 00010000:00010000:2.0:1789302604.491008:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 9 pid: 6695 timeout: 541 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302604.491049:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 173 00000100:00100000:2.0:1789302604.491060:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350177792:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302604.491088:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302604.491113:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 9 pid: 6695 timeout: 541 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.491127:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 9 pid: 6695 timeout: 541 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.491150:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302604.491155:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302604.491164:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.491174:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.491211:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302604.491244:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdd180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371044992:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302604.491344:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05c00 going to sleep for 11 seconds 00010000:00010000:3.0:1789302604.492432:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070a1efe7/0x64fe43095184bfaf lrc: 0/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc5e expref: 9 pid: 6695 timeout: 541 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302604.492453:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db47b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371044864:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302604.495895:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 8 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.495921:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2da00, rc 0) 00000100:00100000:2.0:1789302604.497779:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:18592:x1876219350177664:12345-192.168.203.11@tcp:101:cp.0 Request processed in 14473us (14641us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302604.497789:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 143 00000100:00100000:1.0:1789302604.508093:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350178048 00000100:00100000:1.0:1789302604.508103:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350178048 00000100:00100000:1.0:1789302604.508166:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 123 00000100:00100000:1.0:1789302604.508175:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:18592:x1876219350178048:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:1.0:1789302604.508956:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:18592:x1876219350178048:12345-192.168.203.11@tcp:35:cp.0 Request processed in 779us (971us total) trans 4294967698 rc 0/0 00000100:00100000:1.0:1789302604.508983:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 123 00000100:00100000:1.0:1789302604.571638:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302604.575760:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350178176 00000100:00100000:2.0:1789302604.575776:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350178176 00000100:00100000:2.0:1789302604.575854:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:2.0:1789302604.575865:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:18595:x1876219350178176:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302604.575888:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302604.575909:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bfbd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc6c expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578735:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578758:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578770:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578790:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578798:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578825:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578919:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bfd2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578927:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bfd2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.578934:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bfd2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302604.578968:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00000004:00200000:2.0:1789302604.578972:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302604.578990:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bfd2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579001:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bfd2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579009:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579014:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579021:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579027:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fae00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302604.579031:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579045:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579057:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:2.0:1789302604.579061:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.579065:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579071:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184bfc4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579085:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579091:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579096:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579101:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302604.579104:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579110:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579118:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x4a7b05a0 (ffff960f11d36800) 00010000:00010000:2.0:1789302604.579120:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302604.579123:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579141:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfcb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x4a7b05a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579182:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bfbd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc6c expref: 61 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579197:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbe00/0x64fe43095184bfd2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc6c expref: 60 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.579204:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:2.0:1789302604.579206:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302604.579209:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fbe00, rc 0) 00000100:00100000:2.0:1789302604.579275:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:18595:x1876219350178176:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3410us (7645us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302604.579284:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:1.0:1789302604.583794:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350178304 00000100:00100000:1.0:1789302604.583805:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350178304 00000100:00080000:3.0:1789302604.583848:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302604.583879:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 144 00000100:00100000:1.0:1789302604.583891:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350178304:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302604.583909:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302604.583929:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bd933f66/0x64fe43095184bfd9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc73 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302604.583989:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02322a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371045120:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302604.584224:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf780 going to sleep for 11 seconds 00002000:00010000:1.0:1789302604.587266:0:6696:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb size: 0 -> 158 00002000:00010000:1.0:1789302604.587274:0:6696:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb mtime: 0 -> 1789302604 00002000:00010000:1.0:1789302604.587276:0:6696:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb atime: 0 -> 1789302604 00002000:00010000:1.0:1789302604.587278:0:6696:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xd:0x0] updating lvb ctime: 0 -> 1789302604 00000100:00100000:1.0:1789302604.587301:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02322a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371045120:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302604.587336:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bd933f66/0x64fe43095184bfd9 lrc: 1/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc73 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.587358:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bd933f66/0x64fe43095184bfd9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc73 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302604.587370:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99200, rc 0) 00000100:00100000:1.0:1789302604.587477:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350178304:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3587us (3850us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302604.587724:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 144 00000100:00100000:3.0:1789302604.603193:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302604.603309:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350178432 00000100:00100000:2.0:1789302604.603320:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350178432 00000100:00100000:2.0:1789302604.603384:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 574 00000100:00100000:2.0:1789302604.603396:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:18595:x1876219350178432:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302604.603428:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302604.603448:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bfe0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc7a expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.603560:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfe7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.603575:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfe7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.603601:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfe7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302604.603635:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x22:0x0]: returning size 16384 00000004:00200000:2.0:1789302604.603639:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302604.603665:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfe7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.603692:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bfe0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc7a expref: 62 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.603713:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa400/0x64fe43095184bfe7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x22:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc7a expref: 61 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302604.603723:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:2.0:1789302604.603727:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302604.603730:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa400, rc 0) 00000100:00100000:3.0:1789302604.603965:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:18595:x1876219350178432:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 569us (779us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.603973:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 574 00000100:00100000:3.0:1789302604.617729:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350178560 00000100:00100000:3.0:1789302604.617738:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350178560 00000100:00100000:3.0:1789302604.617792:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 145 00000100:00100000:3.0:1789302604.617802:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2345:x1876219350178560:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302604.617819:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302604.617837:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdbe3ec/0x64fe43095184bfee lrc: 2/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc81 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302604.617898:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371045248:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302604.617983:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302604.633944:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371045248:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302604.633988:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006bdbe3ec/0x64fe43095184bfee lrc: 1/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc81 expref: 9 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.634035:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdbe3ec/0x64fe43095184bfee lrc: 0/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc81 expref: 9 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302604.634049:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c000, rc 0) 00000100:00100000:3.0:1789302604.634788:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2345:x1876219350178560:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 16983us (17166us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302604.634798:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 145 00000100:00100000:1.0:1789302604.649767:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302604.649856:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350178816 00000100:00100000:3.0:1789302604.649867:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350178816 00000100:00100000:3.0:1789302604.649923:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 575 00000100:00100000:3.0:1789302604.649934:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:18595:x1876219350178816:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302604.650370:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].13, 136 off9960 00000100:00100000:3.0:1789302604.650475:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:18595:x1876219350178816:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 543us (717us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302604.650483:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 575 00010000:00010000:3.0:1789302605.453996:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302605.587163:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350179072 00000100:00100000:2.0:1789302605.587174:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350179072 00000100:00100000:2.0:1789302605.587247:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 146 00000100:00100000:2.0:1789302605.587258:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350179072:12345-192.168.203.11@tcp:16:kworker.0 00000100:00100000:2.0:1789302605.587389:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350179072:12345-192.168.203.11@tcp:16:kworker.0 Request processed in 133us (467us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302605.587397:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 146 00000100:00100000:1.0:1789302605.589188:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350178944 00000100:00100000:1.0:1789302605.589202:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350178944 00002000:00100000:1.0:1789302605.589249:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xd:0x0] (0->157) req@ffff960e03982a00 x1876219350178944/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:181/0 lens 648/0 e 0 to 0 dl 1789302616 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302605.589260:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03982a00 with x1876219350178944 ext(0->157) 00002000:00010000:1.0:1789302605.589267:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e03982a00 00000100:00100000:1.0:1789302605.589347:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 52 00000100:00100000:1.0:1789302605.589356:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350178944:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302605.589874:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350178944:12345-192.168.203.11@tcp:4:cp.0 Request processed in 518us (2885us total) trans 4294967326 rc 0/0 00000100:00100000:1.0:1789302605.589882:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 52 00002000:00100000:1.0:1789302605.589889:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xd:0x0] (0->157) req@ffff960e03982a00 x1876219350178944/t4294967326(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:181/0 lens 648/448 e 0 to 0 dl 1789302616 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302605.589897:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03982a00 with x1876219350178944 ext(0->157) 00002000:00010000:1.0:1789302605.589901:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e03982a00 00000040:00080000:2.0:1789302605.602375:0:15732:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302605.602385:0:15732:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302605.602444:0:15732:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 14 count 3 00000040:00080000:2.0:1789302605.602455:0:15732:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789302605.649648:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302605.649659:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302605.649698:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 14 count 3 00000040:00080000:3.0:1789302605.649704:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302605.650031:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371045376:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302605.650123:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302605.665410:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371045376:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302605.668024:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302605.668100:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350179200 00000100:00100000:1.0:1789302605.668110:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350179200 00000100:00100000:1.0:1789302605.668171:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 576 00000100:00100000:1.0:1789302605.668182:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:18606:x1876219350179200:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302605.668217:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xe:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302605.668311:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:18606:x1876219350179200:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 130us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302605.668336:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 576 00000100:00100000:1.0:1789302605.683939:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302605.684703:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350179456 00000100:00100000:2.0:1789302605.684714:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350179456 00000100:00100000:2.0:1789302605.684780:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:2.0:1789302605.684790:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:18606:x1876219350179456:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302605.684817:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302605.684839:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bff5 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc88 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302605.684938:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bffc lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302605.684953:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bffc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302605.684965:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bffc lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302605.684977:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00010000:00010000:2.0:1789302605.685111:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184bff5 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc88 expref: 30 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302605.685130:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bffc lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc88 expref: 29 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302605.685139:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:2.0:1789302605.685142:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302605.685146:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa000, rc 0) 00000100:00100000:2.0:1789302605.685204:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:18606:x1876219350179456:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 416us (1276us total) trans 4294967699 rc 0/0 00000100:00100000:2.0:1789302605.685218:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:2.0:1789302605.692226:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302605.692317:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302605.692742:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350179584 00000100:00100000:3.0:1789302605.692769:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350179584 00000100:00100000:3.0:1789302605.692847:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 19 00000100:00100000:3.0:1789302605.692857:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2345:x1876219350179584:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302605.692874:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302605.692905:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa000/0x64fe43095184bffc lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc88 expref: 29 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.692923:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302605.692928:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302605.692987:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:2345:x1876219350179584:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 132us (770us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302605.692994:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 19 00000100:00100000:1.0:1789302605.703651:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371045504:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302605.703682:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007d643fe4/0x64fe43095184c00a lrc: 3/1,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302605.705835:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350179840 00000100:00100000:1.0:1789302605.705846:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350179840 00010000:00010000:1.0:1789302605.705887:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 10 pid: 6697 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302605.705927:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 174 00000100:00100000:1.0:1789302605.705939:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350179840:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302605.705956:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302605.705980:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 10 pid: 6697 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.705997:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 10 pid: 6697 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.706025:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001928e6b5/0x64fe43095184bfb6 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc65 expref: 10 pid: 6697 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.706044:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302605.706049:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302605.706064:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007d643fe4/0x64fe43095184c00a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.706074:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007d643fe4/0x64fe43095184c00a lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.706112:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302605.706187:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350179840:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 250us (449us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302605.706195:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 174 00000100:00100000:1.0:1789302605.711630:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302605.711692:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350180096 00000100:00100000:3.0:1789302605.711710:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350180096 00000100:00100000:3.0:1789302605.711794:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 578 00000100:00100000:3.0:1789302605.711809:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:18606:x1876219350180096:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302605.711966:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:18606:x1876219350180096:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 160us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302605.711974:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 578 00000100:00100000:3.0:1789302605.720301:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302605.733428:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350180480 00000100:00100000:2.0:1789302605.733439:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350180480 00000100:00100000:2.0:1789302605.733516:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 579 00000100:00100000:2.0:1789302605.733528:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:18606:x1876219350180480:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302605.733638:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:18606:x1876219350180480:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 113us (810us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302605.733646:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 579 00000100:00100000:1.0:1789302605.741156:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302605.750713:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350180736 00000100:00100000:1.0:1789302605.750723:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350180736 00000100:00100000:1.0:1789302605.750781:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 580 00000100:00100000:1.0:1789302605.750792:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:18606:x1876219350180736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302605.750812:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302605.750833:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c018 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc96 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.750869:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.750882:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.750901:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.751006:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c018 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc96 expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.751023:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc96 expref: 29 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.751031:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:1.0:1789302605.751034:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302605.751038:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc98600, rc 0) 00000100:00100000:1.0:1789302605.751111:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:18606:x1876219350180736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 320us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302605.751118:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 580 00000100:00100000:2.0:1789302605.762636:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350180864 00000100:00100000:2.0:1789302605.762647:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350180864 00000100:00100000:2.0:1789302605.762698:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 581 00000100:00100000:2.0:1789302605.762708:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:18606:x1876219350180864:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302605.762736:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xe:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302605.762825:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:18606:x1876219350180864:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 119us (294us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302605.762833:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 581 00000100:00100000:1.0:1789302605.780450:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350181248 00000100:00100000:1.0:1789302605.780460:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350181248 00000100:00100000:3.0:1789302605.798945:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350181376 00000100:00100000:1.0:1789302605.798948:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371045632:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302605.798952:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350181376 00010000:00010000:1.0:1789302605.798975:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.798987:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc96 expref: 30 pid: 5844 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302605.799053:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 175 00000100:00100000:3.0:1789302605.799143:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:2347:x1876219350181376:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302605.799171:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302605.799180:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc96 expref: 30 pid: 5844 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.799192:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc96 expref: 30 pid: 5844 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.799212:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c01f lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc96 expref: 30 pid: 5844 timeout: 542 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.799227:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302605.799233:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,1 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.799244:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.799253:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302605.799386:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302605.799449:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2347:x1876219350181376:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 309us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302605.799456:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 175 00010000:00010000:1.0:1789302605.799834:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.799843:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.800049:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.800057:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.800071:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.800077:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302605.800081:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.800089:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.800100:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:1.0:1789302605.800103:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302605.800107:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 2/0,0 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302605.800113:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c02d lrc: 0/0,0 mode: --/PW res: [0x200000402:0xe:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302605.800260:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:18606:x1876219350181248:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 18835us (20437us total) trans 4294967700 rc 0/0 00000100:00100000:1.0:1789302605.800269:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 582 00000100:00100000:3.0:1789302605.804545:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302605.804615:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350181632 00000100:00100000:2.0:1789302605.804627:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350181632 00000100:00100000:2.0:1789302605.804691:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 124 00000100:00100000:2.0:1789302605.804701:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:18606:x1876219350181632:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302605.804941:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:18606:x1876219350181632:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 241us (412us total) trans 4294967701 rc 0/0 00000100:00100000:2.0:1789302605.804959:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 124 00010000:00010000:1.0:1789302606.476663:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302606.861836:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371046144:0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.861868:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302606.861891:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1789302606.861905:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371045760:0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.861936:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302606.861960:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371046272:0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.861982:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302606.862008:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371045888 00000100:00100000:3.0:1789302606.862012:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371045888 00000100:00100000:3.0:1789302606.862039:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371046016 00000100:00100000:3.0:1789302606.862042:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371046016 00000100:00100000:3.0:1789302606.862057:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371046272 00000100:00100000:3.0:1789302606.862060:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371046272 00000100:00100000:3.0:1789302606.862074:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1789302606.862080:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371045888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.862112:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371045888:12345-0@lo:400:kworker.0 Request processed in 32us (837us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302606.862118:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1789302606.862132:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1789302606.862138:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371046016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.862163:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371046016:12345-0@lo:400:kworker.0 Request processed in 26us (728us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302606.862169:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1789302606.862180:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:3.0:1789302606.862186:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371046272:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.862216:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371046272:12345-0@lo:400:kworker.0 Request processed in 30us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302606.862222:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00080000:3.0:1789302606.862237:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302606.862241:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302606.862259:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371046144 00000100:00100000:3.0:1789302606.862263:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371046144 00000100:00100000:3.0:1789302606.862293:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1789302606.862300:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371046144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.862344:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371046144:12345-0@lo:400:kworker.0 Request processed in 45us (475us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302606.862350:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1789302606.862367:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371045760 00000100:00100000:3.0:1789302606.862370:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371045760 00000100:00100000:3.0:1789302606.862405:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1789302606.862412:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371045760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.862440:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371045760:12345-0@lo:400:kworker.0 Request processed in 30us (504us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302606.862446:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00080000:3.0:1789302606.862481:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302606.862521:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371045760:0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.862535:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371046272:0@lo:400:kworker.0 00000100:00100000:3.0:1789302606.862545:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371046144:0@lo:400:kworker.0 00000100:00100000:2.0:1789302607.117613:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302607.117656:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350181760 00000100:00100000:2.0:1789302607.117665:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350181760 00000100:00100000:2.0:1789302607.117712:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:2.0:1789302607.117722:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350181760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302607.117784:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350181760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 63us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302607.117792:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:3.0:1789302607.122410:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350181888 00000100:00100000:3.0:1789302607.122421:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350181888 00000100:00100000:3.0:1789302607.122489:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:3.0:1789302607.122598:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:2347:x1876219350181888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302607.122666:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:2347:x1876219350181888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 71us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302607.122674:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:3.0:1789302607.122771:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302607.122831:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350182016 00000100:00100000:3.0:1789302607.122835:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350182016 00000100:00100000:3.0:1789302607.123204:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 211 00000100:00100000:3.0:1789302607.123213:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350182016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302607.125038:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2347:x1876219350182016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1820us (2260us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302607.125051:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 211 00000100:00100000:2.0:1789302607.127158:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302607.127286:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302607.127313:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302607.127353:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350182272 00000100:00100000:2.0:1789302607.127362:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350182272 00000100:00100000:2.0:1789302607.127425:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:2.0:1789302607.127435:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350182272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302607.127499:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350182272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 66us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302607.127522:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:3.0:1789302607.131904:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:1.0:1789302607.132217:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302607.372739:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302607.381726:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371046528 00000100:00100000:3.0:1789302607.381742:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371046528 00000100:00100000:3.0:1789302607.383570:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 108 00000100:00100000:3.0:1789302607.383606:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371046528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302607.383646:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 54460416 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302607.383740:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371046528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 151us (4408us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302607.390156:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 108 00010000:00010000:3.0:1789302607.501641:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302607.533373:0:15785:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302607.535070:0:15785:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302607.535163:0:15785:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 14 count 3 00000040:00080000:1.0:1789302607.535169:0:15785:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302607.861298:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302607.862334:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 150 00000100:00100000:2.0:1789302607.862346:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+8:2346:x1876219350182656:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:2.0:1789302607.862365:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302607.862465:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006cddb364/0x64fe43095184c034 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fc9d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302607.862545:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=2s v=(0 0) 00010000:00010000:2.0:1789302607.862551:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302607.862561:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006cddb364/0x64fe43095184c034 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc9d expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302607.862582:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006cddb364/0x64fe43095184c034 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc9d expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302607.862598:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa000, rc 0) 00000100:00100000:2.0:1789302607.863310:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350182656:12345-192.168.203.11@tcp:101:diff.0 Request processed in 961us (2016us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302607.863320:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 150 00000100:00100000:2.0:1789302607.874349:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302607.874547:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350182912 00000100:00100000:3.0:1789302607.874557:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350182912 00000100:00100000:3.0:1789302607.874611:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00100000:3.0:1789302607.874621:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+59:18632:x1876219350182912:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302607.874643:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302607.874848:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c03b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fca4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.874974:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c042 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.874990:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c042 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.875002:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c042 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302607.875013:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00010000:00010000:3.0:1789302607.876490:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c03b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fca4 expref: 62 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.876742:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c042 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fca4 expref: 61 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.876753:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302607.876756:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302607.876759:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:3.0:1789302607.876820:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:18632:x1876219350182912:12345-192.168.203.11@tcp:101:diff.0 Request processed in 2200us (2481us total) trans 4294967702 rc 0/0 00000100:00100000:3.0:1789302607.876829:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00100000:1.0:1789302607.888941:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302607.889019:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302607.889058:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350183168 00000100:00100000:1.0:1789302607.889064:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350183168 00002000:00100000:1.0:1789302607.889125:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xd:0x0] (0->4095) req@ffff960e02322d80 x1876219350183168/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:183/0 lens 488/0 e 0 to 0 dl 1789302618 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302607.889138:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02322d80 with x1876219350183168 ext(0->4095) 00002000:00010000:1.0:1789302607.889143:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e02322d80 00000100:00100000:1.0:1789302607.889164:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 54 00000100:00100000:1.0:1789302607.889172:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350183168:12345-192.168.203.11@tcp:3:diff.0 00000100:00100000:1.0:1789302607.889348:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350183168:12345-192.168.203.11@tcp:3:diff.0 Request processed in 176us (328us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302607.889355:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 54 00002000:00100000:1.0:1789302607.889362:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xd:0x0] (0->4095) req@ffff960e02322d80 x1876219350183168/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:183/0 lens 488/4536 e 0 to 0 dl 1789302618 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302607.889370:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02322d80 with x1876219350183168 ext(0->4095) 00002000:00010000:1.0:1789302607.889374:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e02322d80 00000100:00100000:2.0:1789302607.890556:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350183040 00000100:00100000:2.0:1789302607.890571:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350183040 00000100:00100000:2.0:1789302607.890629:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 20 00000100:00100000:2.0:1789302607.890638:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:2348:x1876219350183040:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302607.890663:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302607.890687:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c042 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fca4 expref: 61 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302607.890708:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:2.0:1789302607.890713:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302607.890757:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:2348:x1876219350183040:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 120us (1818us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302607.890781:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 20 00000100:00100000:1.0:1789302607.898731:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302607.948635:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302607.948885:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350183680 00000100:00100000:3.0:1789302607.948903:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350183680 00000100:00100000:3.0:1789302607.948985:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 584 00000100:00100000:3.0:1789302607.948999:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:18635:x1876219350183680:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302607.949032:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302607.949073:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c050 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcab expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.949206:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2dc00/0x64fe43095184c057 lrc: 3/1,0 mode: --/CR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.949224:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2dc00/0x64fe43095184c057 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.949240:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2dc00/0x64fe43095184c057 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302607.949253:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xe:0x0]: returning size 158 00010000:00010000:3.0:1789302607.949433:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c050 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x22:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcab expref: 64 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.949459:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2dc00/0x64fe43095184c057 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0xe:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcab expref: 63 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302607.949482:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:3.0:1789302607.949486:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:3.0:1789302607.949491:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2dc00, rc 0) 00000100:00100000:3.0:1789302607.949630:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:18635:x1876219350183680:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 633us (1002us total) trans 4294967704 rc 0/0 00000100:00100000:3.0:1789302607.949640:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 584 00000100:00100000:2.0:1789302607.957477:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302608.014410:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302608.014554:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350183936 00000100:00100000:2.0:1789302608.014570:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350183936 00000100:00100000:2.0:1789302608.014652:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 586 00000100:00100000:2.0:1789302608.014663:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:18642:x1876219350183936:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:2.0:1789302608.014723:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 3/1,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014740:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014752:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014777:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014784:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014791:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014796:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fae00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302608.014800:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014807:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014836:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xe:0x0].0x0 (ffff960f14168400) 00010000:00010000:2.0:1789302608.014839:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302608.014844:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 2/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302608.014851:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c06c lrc: 0/0,0 mode: --/PR res: [0x200000402:0xe:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302608.014934:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:18642:x1876219350183936:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 272us (556us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302608.014941:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 586 00010000:00010000:1.0:1789302608.524667:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789302608.984165:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967326 is committed 00010000:00010000:3.0:1789302609.559256:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302610.574088:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302611.597660:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302611.981559:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371047168:0@lo:400:kworker.0 00000100:00100000:2.0:1789302611.981587:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371046784:0@lo:400:kworker.0 00000100:00100000:3.0:1789302611.981639:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302611.981639:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302611.981695:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371047296:0@lo:400:kworker.0 00000100:00100000:2.0:1789302611.981736:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302611.981821:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371047296 00000100:00100000:2.0:1789302611.981830:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371047296 00000100:00100000:2.0:1789302611.981929:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371046784 00000100:00100000:2.0:1789302611.981935:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371046784 00000100:00100000:2.0:1789302611.981977:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1789302611.981990:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371046784:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302611.981993:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371047168:0@lo:400:kworker.0 00000100:00100000:2.0:1789302611.982041:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371046784:12345-0@lo:400:kworker.0 Request processed in 56us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302611.982049:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:3.0:1789302611.982109:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371046784:0@lo:400:kworker.0 00000100:00100000:3.0:1789302611.982126:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371047296:0@lo:400:kworker.0 00000100:00100000:3.0:1789302612.236212:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302612.237409:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302612.237564:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302612.237610:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350184320 00000100:00100000:3.0:1789302612.237623:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350184320 00000100:00100000:3.0:1789302612.239887:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 217 00000100:00100000:3.0:1789302612.239908:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350184320:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302612.240020:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350184320:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 118us (2620us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302612.240029:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 217 00000100:00100000:3.0:1789302612.240070:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350184192 00000100:00100000:3.0:1789302612.240080:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350184192 00000100:00100000:3.0:1789302612.240117:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 258 00000100:00100000:3.0:1789302612.240124:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:2347:x1876219350184192:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302612.240155:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:2347:x1876219350184192:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (2592us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302612.240162:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 258 00000100:00080000:3.0:1789302612.240365:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302612.240371:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302612.240715:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350184064 00000100:00100000:2.0:1789302612.240724:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350184064 00000100:00100000:2.0:1789302612.240767:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:2.0:1789302612.240786:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350184064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302612.240837:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350184064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 52us (4644us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302612.240844:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:2.0:1789302612.242823:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302612.242861:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302612.242890:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302612.242917:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302612.242951:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350184448 00000100:00100000:2.0:1789302612.242963:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350184448 00000100:00100000:2.0:1789302612.243029:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350184704 00000100:00100000:2.0:1789302612.243033:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350184704 00000100:00100000:2.0:1789302612.243056:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350184832 00000100:00100000:2.0:1789302612.243060:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350184832 00000100:00100000:2.0:1789302612.243076:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 218 00000100:00100000:2.0:1789302612.243086:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f050a6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350184448:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302612.243171:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f050a6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350184448:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 86us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302612.243178:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 218 00000100:00100000:2.0:1789302612.243411:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 219 00000100:00100000:2.0:1789302612.243419:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350184704:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302612.243447:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350184704:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (586us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302612.243454:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 219 00000100:00100000:2.0:1789302612.243467:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 220 00000100:00100000:2.0:1789302612.243474:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350184832:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302612.243554:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350184832:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 80us (636us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302612.243561:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 220 00000100:00100000:2.0:1789302612.243585:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350184576 00000100:00100000:2.0:1789302612.243589:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350184576 00000100:00100000:2.0:1789302612.243622:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00100000:2.0:1789302612.243629:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2348:x1876219350184576:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302612.243669:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2348:x1876219350184576:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 39us (778us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302612.243675:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00080000:2.0:1789302612.243697:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302612.243702:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302612.243704:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302612.493918:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302612.493952:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371047552:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302612.494120:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302612.495724:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371047552:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302612.622234:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302613.006565:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371047680:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302613.006687:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302613.006782:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371047680 00000100:00100000:3.0:1789302613.006848:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371047680 00000100:00100000:3.0:1789302613.006916:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:3.0:1789302613.006926:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371047680:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302613.006973:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302613.007045:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371047680:12345-0@lo:13:osp-pre-0-0.0 Request processed in 119us (357us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302613.007052:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:3.0:1789302613.007115:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371047680:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789302613.007645:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302613.063840:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302613.076975:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302613.078843:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350185216 00000100:00100000:3.0:1789302613.078864:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350185216 00000100:00100000:3.0:1789302613.078955:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 126 00000100:00100000:3.0:1789302613.078968:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:18587:x1876219350185216:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302613.079387:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:18587:x1876219350185216:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 420us (2424us total) trans 4294967705 rc 0/0 00000100:00100000:3.0:1789302613.079396:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 126 00010000:00010000:3.0:1789302613.644741:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302614.674493:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1789302614.729264:0:15985:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10b: Archive of non dirty file must work without doing request ========================================================== 08:30:13 (1789302613) 00000100:00100000:3.0:1789302615.075610:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302615.075883:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350185472 00000100:00100000:2.0:1789302615.075893:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350185472 00000100:00100000:2.0:1789302615.075972:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00100000:2.0:1789302615.075984:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:19248:x1876219350185472:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302615.076009:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302615.076030:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095184c073 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcb2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.076120:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.076135:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.076149:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.076184:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.076192:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.076199:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077578:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c088 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077596:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c088 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077607:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c088 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302615.077620:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00010000:00010000:2.0:1789302615.077732:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077741:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077761:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077770:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302615.077775:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077783:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077797:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:2.0:1789302615.077801:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.077806:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077812:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe43095184c07a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077827:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077833:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077838:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077843:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302615.077845:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077860:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077868:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff960f11d36700) 00010000:00010000:2.0:1789302615.077870:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.077874:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.077879:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c081 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.078825:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095184c073 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcb2 expref: 28 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.078846:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c088 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcb2 expref: 27 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.078853:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302615.078855:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.078858:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbea00, rc 0) 00000100:00100000:2.0:1789302615.078972:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19248:x1876219350185472:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2991us (3369us total) trans 4294967706 rc 0/0 00000100:00100000:2.0:1789302615.078981:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00100000:3.0:1789302615.082206:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350185600 00000100:00100000:3.0:1789302615.082216:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350185600 00000100:00100000:3.0:1789302615.082323:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 589 00000100:00100000:3.0:1789302615.082332:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:19249:x1876219350185600:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302615.082361:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302615.082394:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c096 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcb9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082469:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082482:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082493:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082533:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082541:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082548:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082808:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082816:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082822:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082828:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.082832:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082849:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082861:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302615.082863:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.082868:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082873:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c09d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082884:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082891:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082896:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082902:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.082904:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082910:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082916:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x80c094a7 (ffff960f14168000) 00010000:00010000:3.0:1789302615.082919:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.082922:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.082935:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0a4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.083217:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c096 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcb9 expref: 61 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.083228:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302615.083230:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.083234:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c096 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcb9 expref: 61 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.083244:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d400, rc 0) 00000100:00100000:3.0:1789302615.083305:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:19249:x1876219350185600:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 973us (1764us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302615.083312:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 589 00000100:00100000:3.0:1789302615.084449:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.084484:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302615.084556:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350185856 00000100:00100000:1.0:1789302615.084565:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350185856 00000100:00100000:1.0:1789302615.084615:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 21 00000100:00100000:1.0:1789302615.084623:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:2347:x1876219350185856:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302615.084635:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302615.084659:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c088 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcb2 expref: 27 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.084687:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:1.0:1789302615.084691:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302615.084728:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:2347:x1876219350185856:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 105us (242us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302615.084746:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 21 00000100:00100000:2.0:1789302615.085321:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350185728 00000100:00100000:2.0:1789302615.085328:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350185728 00000100:00100000:2.0:1789302615.085372:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 590 00000100:00100000:2.0:1789302615.085379:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:19248:x1876219350185728:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302615.085471:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:19248:x1876219350185728:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 91us (1025us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302615.085478:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 590 00000100:00100000:2.0:1789302615.099553:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.099635:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350186112 00000100:00100000:3.0:1789302615.099646:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350186112 00000100:00100000:3.0:1789302615.099701:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 591 00000100:00100000:3.0:1789302615.099711:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:19249:x1876219350186112:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:3.0:1789302615.099766:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302615.099793:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184beac lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc11 expref: 60 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.099811:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302615.099832:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302615.100040:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.100055:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.100064:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.100110:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 25 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.100125:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 25 pid: 5844 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302615.100140:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371047808:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302615.100235:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302615.104845:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.104988:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371047808:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302615.105015:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302615.105555:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350186240 00000100:00100000:2.0:1789302615.105563:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350186240 00010000:00010000:2.0:1789302615.105602:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 26 pid: 5844 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302615.105642:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 176 00000100:00100000:2.0:1789302615.105651:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:2348:x1876219350186240:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302615.105665:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302615.105671:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 26 pid: 5844 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.105684:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 26 pid: 5844 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.105704:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184bf15 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc2d expref: 26 pid: 5844 timeout: 552 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.105719:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:2.0:1789302615.105724:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.105734:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.105760:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.105783:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302615.105843:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:2348:x1876219350186240:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 193us (1007us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302615.105852:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 176 00010000:00010000:3.0:1789302615.106555:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.106567:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.106714:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.106809:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.106819:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302615.108498:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00010000:00010000:2.0:1789302615.108540:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.108550:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.108557:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.108562:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302615.108567:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.108574:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.108589:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f14168100) 00010000:00010000:2.0:1789302615.108591:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.108596:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.108601:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c0ab lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302615.108628:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000019434a3f reply state 00000000c6e92f92 transno 4294967707 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302615.108636:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302615.108729:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:19249:x1876219350186112:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 9017us (9183us total) trans 4294967707 rc 0/0 00000100:00100000:2.0:1789302615.108737:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 591 00010000:00010000:3.0:1789302615.110493:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.110544:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.110552:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.110557:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.110562:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.110571:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.110588:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x80c094a7 (ffff960f14168000) 00010000:00010000:3.0:1789302615.110591:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.110595:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.110601:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0b2 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302615.119559:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350186368 00000100:00100000:2.0:1789302615.119572:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350186368 00000100:00100000:2.0:1789302615.119715:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 592 00000100:00100000:2.0:1789302615.119726:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+60:19250:x1876219350186368:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302615.119751:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302615.119776:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c0b9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcc0 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.119837:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c0c0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.119849:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c0c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.119861:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c0c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302615.119895:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302615.119898:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302615.119919:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c0c0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.119966:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c0b9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcc0 expref: 62 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.119984:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c0c0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcc0 expref: 61 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.120064:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f11d36700) 00010000:00010000:2.0:1789302615.120068:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302615.120072:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfa00, rc 0) 00000100:00100000:2.0:1789302615.120134:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+61:19250:x1876219350186368:12345-192.168.203.11@tcp:101:bash.0 Request processed in 410us (1941us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302615.120141:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 592 00000100:00100000:1.0:1789302615.122602:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302615.126602:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.126685:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350186624 00000100:00100000:3.0:1789302615.126709:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350186624 00000100:00100000:3.0:1789302615.126764:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 594 00000100:00100000:3.0:1789302615.126772:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+62:19250:x1876219350186624:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302615.126791:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302615.126809:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0e3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcce expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126869:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126881:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126892:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126906:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126914:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126920:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126985:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.126993:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127003:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127016:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.127020:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127027:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127039:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:3.0:1789302615.127042:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.127047:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127054:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c0ea lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127066:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127071:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127077:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127083:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.127085:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127090:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127108:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff960f14168000) 00010000:00010000:3.0:1789302615.127111:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.127114:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127121:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c0f1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127155:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0e3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcce expref: 63 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127166:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:3.0:1789302615.127168:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.127171:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c0e3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcce expref: 63 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.127181:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c400, rc 0) 00000100:00100000:3.0:1789302615.127246:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+62:19250:x1876219350186624:12345-192.168.203.11@tcp:101:bash.0 Request processed in 474us (653us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302615.127252:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 594 00000100:00100000:1.0:1789302615.136191:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302615.147989:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302615.148216:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350186880 00000100:00100000:2.0:1789302615.148228:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350186880 00000100:00100000:2.0:1789302615.148307:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 596 00000100:00100000:2.0:1789302615.148317:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:19251:x1876219350186880:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302615.148348:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302615.148371:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c106 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcdc expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148561:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148576:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148588:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148622:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148630:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148636:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148729:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe600/0x64fe43095184c11b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148738:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe600/0x64fe43095184c11b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148746:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe600/0x64fe43095184c11b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302615.148774:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00000004:00200000:2.0:1789302615.148778:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302615.148796:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe600/0x64fe43095184c11b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148808:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148815:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148822:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148839:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302615.148843:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148851:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148865:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f11d36700) 00010000:00010000:2.0:1789302615.148868:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.148874:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148880:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c10d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148893:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148900:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148905:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148910:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302615.148913:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148918:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148936:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x80c094a7 (ffff960f11d36d00) 00010000:00010000:2.0:1789302615.148940:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.148943:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148949:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c114 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x80c094a7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.148980:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c106 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcdc expref: 28 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.149009:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe600/0x64fe43095184c11b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcdc expref: 27 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.149017:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:2.0:1789302615.149019:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302615.149022:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe600, rc 0) 00000100:00100000:1.0:1789302615.152408:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302615.158009:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.158196:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350187136 00000100:00100000:3.0:1789302615.158206:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350187136 00000100:00100000:3.0:1789302615.158262:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 598 00000100:00100000:3.0:1789302615.158272:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19251:x1876219350187136:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302615.158293:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302615.158313:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c137 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcea expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158414:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158429:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158440:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158455:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158462:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158468:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158697:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158706:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158713:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158719:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.158723:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158742:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158754:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:3.0:1789302615.158757:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.158762:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158768:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c13e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158780:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158786:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158791:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158796:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.158798:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158803:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158808:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff960f14168000) 00010000:00010000:3.0:1789302615.158811:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.158814:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158829:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c145 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158862:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c137 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcea expref: 28 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158872:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:3.0:1789302615.158873:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.158877:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c137 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcea expref: 28 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.158885:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c400, rc 0) 00000100:00100000:3.0:1789302615.158987:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19251:x1876219350187136:12345-192.168.203.11@tcp:101:rm.0 Request processed in 715us (985us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302615.158994:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 598 00000100:00100000:3.0:1789302615.172654:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302615.172820:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350187264 00000100:00100000:2.0:1789302615.172831:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350187264 00000100:00100000:2.0:1789302615.172896:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 599 00000100:00100000:2.0:1789302615.172906:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19252:x1876219350187264:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302615.172926:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302615.172945:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c14c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcf1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173024:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173058:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173069:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173087:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173094:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173101:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173150:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173158:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173164:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173170:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302615.173174:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173181:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173193:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:2.0:1789302615.173202:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.173208:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173214:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c153 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173229:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173234:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173240:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173244:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302615.173247:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173252:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173404:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff960f11d36d00) 00010000:00010000:2.0:1789302615.173407:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.173410:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173416:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c15a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173449:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c14c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcf1 expref: 28 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173467:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:2.0:1789302615.173469:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302615.173473:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c14c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcf1 expref: 28 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.173483:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbec00, rc 0) 00000100:00100000:2.0:1789302615.173778:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19252:x1876219350187264:12345-192.168.203.11@tcp:101:cp.0 Request processed in 873us (1131us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302615.173786:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 599 00000100:00100000:3.0:1789302615.177129:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350187392 00000100:00100000:3.0:1789302615.177139:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350187392 00000100:00100000:3.0:1789302615.177201:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 600 00000100:00100000:3.0:1789302615.177211:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19252:x1876219350187392:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302615.177231:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302615.177251:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c161 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcf8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177346:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177381:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177392:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177416:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177424:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177432:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177484:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177491:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177497:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177530:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.177535:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177542:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177553:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:3.0:1789302615.177563:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.177568:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177574:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c168 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177587:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177593:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177599:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177617:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302615.177620:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177626:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177634:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0xb5b5b627 (ffff960f14168000) 00010000:00010000:3.0:1789302615.177636:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.177639:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177645:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c16f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177679:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c161 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcf8 expref: 28 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177696:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:3.0:1789302615.177699:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302615.177703:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c161 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcf8 expref: 28 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.177712:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c400, rc 0) 00000100:00100000:3.0:1789302615.178887:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19252:x1876219350187392:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1670us (1828us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302615.178898:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 600 00000100:00100000:1.0:1789302615.182944:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350187648 00000100:00100000:1.0:1789302615.182954:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350187648 00000100:00100000:1.0:1789302615.183022:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 601 00000100:00100000:1.0:1789302615.183031:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19252:x1876219350187648:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302615.183051:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302615.183074:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c176 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fcff expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.183192:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.183206:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.183218:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.183250:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c184 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.183269:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c184 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.183276:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c184 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302615.184818:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xd:0x0], index 1: 0 00010000:00010000:1.0:1789302615.185072:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184c18b lrc: 3/1,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185087:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184c18b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185098:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184c18b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185135:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185142:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185148:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185154:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302615.185158:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185177:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185189:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:1.0:1789302615.185192:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302615.185197:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185203:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c17d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302615.185216:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006bb8e5ae reply state 0000000055211c46 transno 4294967708 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c184 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x25:0x0].0xb5b5b627 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302615.185225:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302615.185766:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c176 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcff expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185785:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99400/0x64fe43095184c18b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcff expref: 29 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.185792:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff960f421e9500) 00010000:00010000:1.0:1789302615.185794:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302615.185796:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99400, rc 0) 00000100:00100000:1.0:1789302615.185862:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:19252:x1876219350187648:12345-192.168.203.11@tcp:101:cp.0 Request processed in 2831us (3032us total) trans 4294967708 rc 0/0 00000100:00100000:1.0:1789302615.185871:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 601 00000100:00100000:2.0:1789302615.192870:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350187776 00000100:00100000:2.0:1789302615.192880:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350187776 00000100:00100000:2.0:1789302615.192967:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 151 00000100:00100000:2.0:1789302615.192984:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350187776:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302615.193008:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302615.193032:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd06 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302615.193059:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302615.193080:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=452s v=(0 0) 00010000:00010000:2.0:1789302615.193088:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302615.193097:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 8 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.193115:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 8 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.193147:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf200, rc 0) 00000100:00100000:2.0:1789302615.193213:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350187776:12345-192.168.203.11@tcp:101:cp.0 Request processed in 236us (485us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302615.193220:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 151 00000100:00100000:1.0:1789302615.199449:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302615.199535:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350187904 00000100:00100000:2.0:1789302615.199545:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350187904 00000100:00100000:2.0:1789302615.199611:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 152 00000100:00100000:2.0:1789302615.199622:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:19252:x1876219350187904:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302615.199641:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302615.199660:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 2/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd0d expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.199691:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 9 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.199700:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302615.199704:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302615.199709:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 9 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.199741:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 9 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302615.199755:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 9 pid: 8571 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302615.199788:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371047936:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302615.199889:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80f00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302615.203082:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.203189:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdf800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371047936:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302615.203223:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 3/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 9 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.203244:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfe00, rc 0) 00000100:00100000:1.0:1789302615.203345:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350188032 00000100:00100000:1.0:1789302615.203366:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350188032 00010000:00010000:1.0:1789302615.203398:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 10 pid: 8571 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302615.203438:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:1.0:1789302615.203447:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350188032:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302615.203460:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302615.203482:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 10 pid: 8571 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.203545:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 10 pid: 8571 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.203570:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000007814f44/0x64fe43095184c199 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd06 expref: 10 pid: 8571 timeout: 552 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.203585:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302615.203589:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302615.203596:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 9 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.203607:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 9 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302615.203641:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 9 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302615.203711:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:19252:x1876219350187904:12345-192.168.203.11@tcp:101:cp.0 Request processed in 4090us (4269us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302615.203719:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 152 00000100:00100000:1.0:1789302615.203792:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095af800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5830:1876219371048064:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302615.203843:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0958f780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302615.209372:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095af800 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5830:1876219371048064:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302615.209397:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302615.209464:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350188032:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 6017us (6388us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302615.209475:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:2.0:1789302615.210235:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350188288 00000100:00100000:2.0:1789302615.210245:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350188288 00000100:00100000:2.0:1789302615.210297:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 127 00000100:00100000:2.0:1789302615.210306:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19252:x1876219350188288:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302615.211031:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:19252:x1876219350188288:12345-192.168.203.11@tcp:35:cp.0 Request processed in 724us (1724us total) trans 4294967709 rc 0/0 00000100:00100000:2.0:1789302615.211039:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 127 00000100:00100000:2.0:1789302615.274081:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350188416 00000100:00100000:2.0:1789302615.274091:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350188416 00000100:00100000:3.0:1789302615.282887:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350188544 00000100:00100000:3.0:1789302615.282898:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350188544 00000100:00100000:3.0:1789302615.283006:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 153 00000100:00100000:3.0:1789302615.283016:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350188544:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302615.283032:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302615.283050:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000178fc17/0x64fe43095184c1c3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd1b expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302615.283125:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371048192:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302615.283232:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea540 going to sleep for 11 seconds 00002000:00010000:3.0:1789302615.285931:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1789302615.285939:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb mtime: 0 -> 1789302614 00002000:00010000:3.0:1789302615.285941:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb atime: 0 -> 1789302614 00002000:00010000:3.0:1789302615.285944:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0xd:0x0] updating lvb ctime: 0 -> 1789302614 00000100:00100000:3.0:1789302615.285967:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371048192:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302615.286002:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000178fc17/0x64fe43095184c1c3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd1b expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.286025:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000178fc17/0x64fe43095184c1c3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd1b expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.286039:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c400, rc 0) 00000100:00100000:3.0:1789302615.286096:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350188544:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3081us (3304us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302615.286104:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 153 00000100:00100000:1.0:1789302615.289322:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.290655:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350188672 00000100:00100000:3.0:1789302615.290667:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350188672 00000100:00100000:3.0:1789302615.290733:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 603 00000100:00100000:3.0:1789302615.290743:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:19255:x1876219350188672:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302615.290763:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302615.290788:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c1ca lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd22 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.290882:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c1d1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.290897:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c1d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.290909:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c1d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302615.290942:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x25:0x0]: returning size 16384 00000004:00200000:3.0:1789302615.290945:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302615.290962:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c1d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.291155:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c1ca lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd22 expref: 65 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.291188:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c000/0x64fe43095184c1d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x25:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd22 expref: 64 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302615.291197:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x25:0x0].0x0 (ffff960f02dfd500) 00010000:00010000:3.0:1789302615.291200:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302615.291203:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c000, rc 0) 00000100:00100000:3.0:1789302615.291481:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+64:19255:x1876219350188672:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 738us (2166us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302615.291488:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 603 00000100:00100000:2.0:1789302615.298487:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350188800 00000100:00100000:2.0:1789302615.298670:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350188800 00000100:00100000:2.0:1789302615.298774:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 154 00000100:00100000:2.0:1789302615.298785:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350188800:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302615.298803:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302615.298832:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000070d624f2/0x64fe43095184c1d8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd29 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302615.298898:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcddc00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371048320:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302615.298988:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302615.313706:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302615.313825:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350189056 00000100:00100000:3.0:1789302615.313836:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350189056 00000100:00100000:3.0:1789302615.313914:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 604 00000100:00100000:3.0:1789302615.313925:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+64:19255:x1876219350189056:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302615.314190:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].14, 136 off10096 00000100:00100000:3.0:1789302615.314275:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+64:19255:x1876219350189056:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 352us (577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302615.314284:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 604 00010000:00010000:3.0:1789302615.693071:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302616.047118:0:16034:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302616.047126:0:16034:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302616.047163:0:16034:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 3 00000040:00080000:3.0:1789302616.047168:0:16034:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:2.0:1789302616.082098:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302616.082108:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302616.082159:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 3 00000040:00080000:2.0:1789302616.082165:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302616.082418:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371048448:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302616.085953:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302616.086005:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371048448:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302616.104672:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302616.104707:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350189184 00000100:00100000:3.0:1789302616.104716:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350189184 00000100:00100000:3.0:1789302616.104764:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 605 00000100:00100000:3.0:1789302616.104773:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:19265:x1876219350189184:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302616.104807:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0xf:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302616.104901:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:19265:x1876219350189184:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 128us (237us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302616.104909:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 605 00000100:00100000:1.0:1789302616.121748:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302616.121871:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350189440 00000100:00100000:2.0:1789302616.121881:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350189440 00000100:00100000:2.0:1789302616.121942:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 606 00000100:00100000:2.0:1789302616.121952:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:19265:x1876219350189440:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302616.121972:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302616.121993:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c1df lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd30 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.122088:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c1e6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.122102:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c1e6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.122116:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c1e6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302616.124236:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0xf:0x0]: returning size 158 00010000:00010000:2.0:1789302616.124868:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c1df lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd30 expref: 33 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.124889:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c1e6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd30 expref: 32 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.124915:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff960f421e9500) 00010000:00010000:2.0:1789302616.124918:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302616.124922:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302616.124988:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:19265:x1876219350189440:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 3035us (3248us total) trans 4294967710 rc 0/0 00000100:00100000:2.0:1789302616.124996:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 606 00000100:00100000:2.0:1789302616.134721:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302616.134757:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302616.135286:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350189696 00000100:00100000:1.0:1789302616.135296:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350189696 00000100:00100000:1.0:1789302616.135363:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 22 00000100:00100000:1.0:1789302616.135374:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:2346:x1876219350189696:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302616.135390:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302616.135428:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c1e6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd30 expref: 32 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.135449:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff960f421e9500) 00010000:00010000:1.0:1789302616.135454:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302616.135525:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2346:x1876219350189696:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 153us (766us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302616.135533:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 22 00000100:00100000:3.0:1789302616.136155:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350189568 00000100:00100000:3.0:1789302616.136163:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350189568 00000100:00100000:3.0:1789302616.136213:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 155 00000100:00100000:3.0:1789302616.136221:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350189568:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302616.136257:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.136288:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 8 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.136296:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xd:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302616.136301:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302616.136305:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 4/1,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.136348:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 8 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.136362:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 8 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302616.136375:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371048576:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302616.136455:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302616.140030:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371048576:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302616.140057:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302616.140599:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302616.140640:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350189824 00000100:00100000:1.0:1789302616.140647:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350189824 00002000:00100000:1.0:1789302616.140690:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xd:0x0] (0->157) req@ffff960f095ace00 x1876219350189824/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:192/0 lens 648/0 e 0 to 0 dl 1789302627 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302616.140701:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f095ace00 with x1876219350189824 ext(0->157) 00010000:00010000:1.0:1789302616.140708:0:6700:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 9 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.140722:0:6700:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 9 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302616.140730:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f095ace00 00000100:00100000:1.0:1789302616.140755:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 3 00000100:00100000:1.0:1789302616.140764:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350189824:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302616.141205:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350189824:12345-192.168.203.11@tcp:4:cp.0 Request processed in 441us (607us total) trans 4294967327 rc 0/0 00000100:00100000:1.0:1789302616.141213:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 3 00002000:00100000:1.0:1789302616.141220:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xd:0x0] (0->157) req@ffff960f095ace00 x1876219350189824/t4294967327(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:192/0 lens 648/448 e 0 to 0 dl 1789302627 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302616.141229:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f095ace00 with x1876219350189824 ext(0->157) 00010000:00010000:1.0:1789302616.141236:0:6700:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 11 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.141248:0:6700:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 11 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302616.141267:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f095ace00 00000100:00100000:3.0:1789302616.147726:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302616.147761:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350189952 00000100:00100000:3.0:1789302616.147770:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350189952 00010000:00010000:3.0:1789302616.147807:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 11 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302616.147859:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 178 00000100:00100000:3.0:1789302616.147869:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350189952:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302616.147885:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302616.147908:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 11 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.147923:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 11 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.147947:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c1a0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xd:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd0d expref: 11 pid: 13439 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.147965:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302616.147970:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302616.147975:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.147986:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148014:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302616.148062:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350189952:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 194us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302616.148070:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 178 00010000:00010000:3.0:1789302616.148105:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148115:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148138:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148147:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148157:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148165:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302616.148169:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148180:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148193:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302616.148201:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f36a010c/0x64fe43095184c1f4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302616.148259:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350189568:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 12039us (13548us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302616.148267:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 155 00000001:00080000:2.0:1789302616.151783:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967708 is committed 00000001:00080000:2.0:1789302616.151800:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967709 is committed 00000001:00080000:2.0:1789302616.151805:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967707 is committed 00000100:00100000:2.0:1789302616.165850:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350190208 00000100:00100000:2.0:1789302616.165860:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350190208 00000100:00100000:2.0:1789302616.165921:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 607 00000100:00100000:2.0:1789302616.165931:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19265:x1876219350190208:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302616.166032:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19265:x1876219350190208:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 102us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302616.166040:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 607 00000100:00100000:3.0:1789302616.178154:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302616.178234:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350190336 00000100:00100000:1.0:1789302616.178244:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350190336 00000100:00100000:1.0:1789302616.178297:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 156 00000100:00100000:1.0:1789302616.178307:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350190336:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302616.178324:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302616.178357:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000008414164b/0x64fe43095184c1fb lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd37 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302616.178382:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xd:0x0] initial LVB size: 158, mtime: 0x6aa69756, atime: 0x0, ctime: 0x6aa69756, blocks: 0x1 00010000:00010000:1.0:1789302616.178400:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=453s v=(0 0) 00010000:00010000:1.0:1789302616.178404:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302616.178494:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000008414164b/0x64fe43095184c1fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd37 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.178621:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000008414164b/0x64fe43095184c1fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd37 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.178634:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc98400, rc 0) 00000100:00100000:1.0:1789302616.179486:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350190336:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1177us (1337us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302616.179496:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 156 00000100:00100000:3.0:1789302616.196401:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302616.196815:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350190592 00000100:00100000:1.0:1789302616.196825:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350190592 00000100:00100000:1.0:1789302616.196921:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:1.0:1789302616.196931:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19265:x1876219350190592:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302616.197078:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19265:x1876219350190592:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 148us (687us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302616.197087:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:2.0:1789302616.200022:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302616.200104:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350190720 00000100:00100000:1.0:1789302616.200113:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350190720 00002000:00100000:1.0:1789302616.200157:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (0->4095) req@ffff960f3fcddc00 x1876219350190720/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:192/0 lens 488/0 e 0 to 0 dl 1789302627 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302616.200168:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcddc00 with x1876219350190720 ext(0->4095) 00002000:00010000:1.0:1789302616.200175:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fcddc00 00000100:00100000:1.0:1789302616.200230:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 55 00000100:00100000:1.0:1789302616.200239:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350190720:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302616.200424:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcddc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350190720:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 185us (409us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302616.200432:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 55 00002000:00100000:1.0:1789302616.200439:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xd:0x0] (0->4095) req@ffff960f3fcddc00 x1876219350190720/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:192/0 lens 488/4536 e 0 to 0 dl 1789302627 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302616.200448:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcddc00 with x1876219350190720 ext(0->4095) 00002000:00010000:1.0:1789302616.200451:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fcddc00 00000100:00100000:2.0:1789302616.225187:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302616.225222:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350190848 00000100:00100000:2.0:1789302616.225231:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350190848 00000100:00100000:2.0:1789302616.225297:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 609 00000100:00100000:2.0:1789302616.225306:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19265:x1876219350190848:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302616.225327:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302616.225364:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf400/0x64fe43095184c202 lrc: 2/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd3e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.225416:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 3/1,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.225430:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.225442:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.225570:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf400/0x64fe43095184c202 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 29 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.225586:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 28 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.225595:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff960f421e9500) 00010000:00010000:2.0:1789302616.225598:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302616.225602:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe800, rc 0) 00000100:00100000:2.0:1789302616.225653:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:19265:x1876219350190848:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 348us (476us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302616.225661:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 609 00000100:00100000:3.0:1789302616.237343:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302616.257379:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302616.263011:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350191360 00000100:00100000:1.0:1789302616.263022:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350191360 00000100:00100000:1.0:1789302616.263072:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 611 00000100:00100000:1.0:1789302616.263082:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:19265:x1876219350191360:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1789302616.263173:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.263191:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 28 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.263200:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 4/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.263234:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 28 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.263247:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 28 pid: 5844 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302616.263260:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371048704:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302616.263799:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1e40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302616.270940:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ac000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371048704:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302616.270968:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302616.275115:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302616.276197:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350191488 00000100:00100000:1.0:1789302616.276208:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350191488 00010000:00010000:1.0:1789302616.276321:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 29 pid: 5844 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302616.276378:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 179 00000100:00100000:1.0:1789302616.276389:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2348:x1876219350191488:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302616.276420:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302616.276428:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 29 pid: 5844 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.276442:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 29 pid: 5844 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.276465:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c209 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd3e expref: 29 pid: 5844 timeout: 553 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.276655:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff960f421e9500) 00010000:00010000:1.0:1789302616.276660:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,1 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.276671:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.276679:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302616.276748:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302616.276815:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:2348:x1876219350191488:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 427us (1706us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302616.276823:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 179 00010000:00010000:2.0:1789302616.277197:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277206:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277927:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277938:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277945:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277950:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302616.277955:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277962:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277974:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xf:0x0].0x0 (ffff960f421e9500) 00010000:00010000:2.0:1789302616.277977:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302616.277981:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 2/0,0 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302616.277987:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c217 lrc: 0/0,0 mode: --/PW res: [0x200000402:0xf:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302616.278165:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:19265:x1876219350191360:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 15082us (15398us total) trans 4294967711 rc 0/0 00000100:00100000:2.0:1789302616.278174:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 611 00000040:00080000:3.0:1789302616.278467:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302616.278472:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302616.279160:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:3.0:1789302616.279172:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302616.287679:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302616.287779:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350191744 00000100:00100000:2.0:1789302616.287789:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350191744 00000100:00100000:2.0:1789302616.287846:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 128 00000100:00100000:2.0:1789302616.287873:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:19265:x1876219350191744:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302616.288115:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19265:x1876219350191744:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 244us (443us total) trans 4294967712 rc 0/0 00000100:00100000:2.0:1789302616.288122:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 128 00010000:00010000:2.0:1789302616.716907:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302617.100893:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371048832:0@lo:400:kworker.0 00000100:00100000:2.0:1789302617.100987:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302617.102343:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371049216:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.102395:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302617.102442:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371049088:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.102467:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302617.102535:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302617.102542:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371048960:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.102566:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302617.102595:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371049216 00000100:00100000:1.0:1789302617.102601:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371049216 00000100:00100000:1.0:1789302617.102662:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:1.0:1789302617.102672:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371049216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.102714:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371049216:12345-0@lo:400:kworker.0 Request processed in 45us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.102741:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00080000:1.0:1789302617.102763:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302617.102792:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371049216:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.102813:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371049088 00000100:00100000:1.0:1789302617.102816:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371049088 00000100:00100000:1.0:1789302617.102844:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371048960 00000100:00100000:1.0:1789302617.102847:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371048960 00000100:00100000:1.0:1789302617.102864:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:1.0:1789302617.102870:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371049088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.102904:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371049088:12345-0@lo:400:kworker.0 Request processed in 33us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.102910:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:1.0:1789302617.102923:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:1.0:1789302617.102928:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371048960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.102966:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371048960:12345-0@lo:400:kworker.0 Request processed in 38us (399us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.102972:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00080000:1.0:1789302617.102985:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302617.103013:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371048960:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.103033:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371049088:0@lo:400:kworker.0 00000100:00100000:2.0:1789302617.103543:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371048832:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.103687:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302617.103744:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302617.103752:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371049472:0@lo:400:kworker.0 00000100:00100000:3.0:1789302617.103781:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302617.103808:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371049472 00000100:00100000:3.0:1789302617.103812:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371049472 00000100:00100000:3.0:1789302617.103843:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1789302617.103848:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371049472:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302617.103879:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371049472:12345-0@lo:400:kworker.0 Request processed in 31us (97us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302617.103885:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00080000:3.0:1789302617.103899:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302617.103916:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371049472:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.104224:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371049344:0@lo:400:kworker.0 00000100:00100000:1.0:1789302617.104254:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302617.105492:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371049344 00000100:00100000:2.0:1789302617.105512:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371049344 00000100:00100000:2.0:1789302617.106006:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1789302617.106017:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371049344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302617.106100:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371049344:12345-0@lo:400:kworker.0 Request processed in 86us (1844us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302617.106106:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00080000:3.0:1789302617.106458:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302617.356887:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302617.356945:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302617.356978:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302617.357012:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302617.357049:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302617.357086:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350192000 00000100:00100000:1.0:1789302617.357100:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350192000 00000100:00100000:1.0:1789302617.357177:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350192384 00000100:00100000:1.0:1789302617.357181:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350192384 00000100:00100000:1.0:1789302617.357210:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:1.0:1789302617.357222:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:2345:x1876219350192000:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302617.357331:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:2345:x1876219350192000:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 101us (449us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.357340:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:1.0:1789302617.357361:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 264 00000100:00100000:1.0:1789302617.357369:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2346:x1876219350192384:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302617.357404:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2346:x1876219350192384:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (390us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.357410:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 264 00000100:00100000:1.0:1789302617.357432:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350192128 00000100:00100000:1.0:1789302617.357452:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350192128 00000100:00100000:1.0:1789302617.357485:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350192640 00000100:00100000:1.0:1789302617.357488:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350192640 00000100:00100000:1.0:1789302617.357560:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 223 00000100:00100000:1.0:1789302617.357572:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350192128:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302617.357603:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350192128:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (624us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.357610:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 223 00000100:00100000:1.0:1789302617.357621:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 224 00000100:00100000:1.0:1789302617.357627:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350192640:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302617.357648:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350192640:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (600us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.357654:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 224 00000100:00080000:1.0:1789302617.357670:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302617.357674:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302617.357677:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302617.358061:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350191872 00000100:00100000:3.0:1789302617.358067:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350191872 00000100:00100000:3.0:1789302617.358105:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:3.0:1789302617.358114:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350191872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302617.358152:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350191872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 40us (1206us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302617.358159:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:2.0:1789302617.613805:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371049600 00000100:00100000:2.0:1789302617.613819:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371049600 00000100:00100000:2.0:1789302617.613894:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 112 00000100:00100000:2.0:1789302617.613908:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371049600:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302617.613937:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 57868288 pending 1703936 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302617.614029:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371049600:12345-0@lo:13:osp-pre-1-0.0 Request processed in 124us (1061us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302617.614037:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 112 00000100:00100000:2.0:1789302617.614221:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371049600:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302617.741817:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302617.920065:0:16085:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302617.920075:0:16085:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302617.920134:0:16085:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 3 00000040:00080000:1.0:1789302617.920140:0:16085:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302617.973893:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350192768 00000100:00100000:1.0:1789302617.973904:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350192768 00000100:00100000:1.0:1789302617.973961:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 158 00000100:00100000:1.0:1789302617.973971:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350192768:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302617.973990:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302617.974008:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df9f3ea1/0x64fe43095184c21e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd45 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302617.974051:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302617.974055:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xd:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302617.974065:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000df9f3ea1/0x64fe43095184c21e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd45 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302617.974082:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df9f3ea1/0x64fe43095184c21e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xd:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd45 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302617.974095:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc98e00, rc 0) 00000100:00100000:1.0:1789302617.974158:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2345:x1876219350192768:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 188us (380us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302617.974166:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 158 00000100:00100000:1.0:1789302617.990973:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302617.991054:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193024 00000100:00100000:3.0:1789302617.991064:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193024 00000100:00100000:3.0:1789302617.991126:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:3.0:1789302617.991136:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:19274:x1876219350193024:12345-192.168.203.11@tcp:58:lfs.0 00000100:00100000:3.0:1789302617.991267:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:19274:x1876219350193024:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 131us (304us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302617.991273:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:3.0:1789302618.124789:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371049728:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302618.124859:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000040:00080000:3.0:1789302618.321196:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302618.321206:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302618.321669:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 2 00000040:00080000:3.0:1789302618.321678:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789302618.767493:0:16136:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302618.767517:0:16136:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302618.767558:0:16136:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 2 00000040:00080000:3.0:1789302618.767565:0:16136:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302618.770176:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302619.793665:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302620.813229:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302621.244995:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967712 is committed 00010000:00010000:2.0:1789302621.836731:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302622.221930:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371050112:0@lo:400:kworker.0 00000100:00100000:3.0:1789302622.222066:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302622.222135:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789302622.222140:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371049984:0@lo:400:kworker.0 00000100:00100000:3.0:1789302622.222160:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302622.222179:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371050496:0@lo:400:kworker.0 00000100:00100000:3.0:1789302622.222202:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302622.222245:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371050112 00000100:00100000:3.0:1789302622.222253:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371050112 00000100:00100000:3.0:1789302622.222316:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371049984 00000100:00100000:3.0:1789302622.222319:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371049984 00000100:00100000:3.0:1789302622.222340:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:3.0:1789302622.222348:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371050112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302622.222398:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371050112:12345-0@lo:400:kworker.0 Request processed in 51us (332us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302622.222404:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:3.0:1789302622.222423:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1789302622.222429:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371049984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302622.222454:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371049984:12345-0@lo:400:kworker.0 Request processed in 26us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302622.222459:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1789302622.222478:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371050496 00000100:00100000:3.0:1789302622.222481:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371050496 00000100:00100000:3.0:1789302622.222535:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789302622.222541:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371050496:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789302622.222629:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302622.222634:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302622.223224:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371050240 00000100:00100000:1.0:1789302622.223229:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371050240 00000100:00100000:1.0:1789302622.223261:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:1.0:1789302622.223268:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371050240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302622.223327:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371050240:12345-0@lo:400:kworker.0 Request processed in 61us (586us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302622.223333:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00080000:1.0:1789302622.223353:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302622.223356:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302622.223371:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371050240:0@lo:400:kworker.0 00000100:00100000:3.0:1789302622.224950:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371050496:12345-0@lo:400:kworker.0 Request processed in 2401us (2739us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302622.224973:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789302622.479603:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302622.479655:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302622.479691:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302622.479727:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302622.479744:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302622.479761:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302622.479783:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302622.479813:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193536 00000100:00100000:3.0:1789302622.479822:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193536 00000100:00100000:3.0:1789302622.479873:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193792 00000100:00100000:3.0:1789302622.479877:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193792 00000100:00100000:3.0:1789302622.479892:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193408 00000100:00100000:3.0:1789302622.479894:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193408 00000100:00100000:3.0:1789302622.479919:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193920 00000100:00100000:3.0:1789302622.479921:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193920 00000100:00100000:3.0:1789302622.479937:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00100000:3.0:1789302622.479948:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350193536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302622.480166:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193152 00000100:00100000:1.0:1789302622.480171:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193152 00000100:00100000:3.0:1789302622.480175:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350193536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (358us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302622.480184:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00100000:1.0:1789302622.480200:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:3.0:1789302622.480201:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 230 00000100:00100000:3.0:1789302622.480207:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350193792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302622.480207:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350193152:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302622.480398:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 232 00000100:00100000:2.0:1789302622.480408:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350193920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302622.480443:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2347:x1876219350193792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (541us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302622.480451:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 230 00000100:00100000:3.0:1789302622.480485:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193280 00000100:00100000:3.0:1789302622.480489:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193280 00000100:00100000:3.0:1789302622.480537:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350193664 00000100:00100000:3.0:1789302622.480541:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350193664 00000100:00100000:3.0:1789302622.480585:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 268 00000100:00100000:3.0:1789302622.480592:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:2347:x1876219350193280:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302622.484012:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350193152:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 3788us (4267us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302622.484028:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:2.0:1789302622.485749:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350193920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5332us (5954us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302622.485762:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 232 00000100:00100000:3.0:1789302622.486633:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:2347:x1876219350193280:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 6037us (7035us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302622.486646:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 268 00010000:00010000:3.0:1789302622.863795:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302623.248623:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371050752:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302623.248693:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302623.256630:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371050752:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302623.669820:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302623.677046:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350194304 00000100:00100000:2.0:1789302623.677062:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350194304 00000100:00100000:2.0:1789302623.677150:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 129 00000100:00100000:2.0:1789302623.677162:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+26:19248:x1876219350194304:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302623.677459:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19248:x1876219350194304:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 298us (543us total) trans 4294967713 rc 0/0 00000100:00100000:2.0:1789302623.677467:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 129 00010000:00010000:2.0:1789302623.886412:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1789302625.214993:0:16336:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10c: Check forbidden archive ========== 08:30:23 (1789302623) 00000100:00100000:3.0:1789302625.507226:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:1.0:1789302625.512758:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302625.519244:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302625.519393:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302625.519497:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350194816 00000100:00100000:3.0:1789302625.519533:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350194816 00000100:00100000:3.0:1789302625.519624:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 23 00000100:00100000:3.0:1789302625.519642:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2346:x1876219350194816:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302625.519663:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302625.519698:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c23a lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd4c expref: 29 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.519723:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302625.519729:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302625.519799:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:2346:x1876219350194816:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 165us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302625.519807:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 23 00000100:00100000:3.0:1789302625.521492:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302625.526665:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350195072 00000100:00100000:1.0:1789302625.526688:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350195072 00000100:00080000:3.0:1789302625.526826:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302625.526857:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 616 00000100:00100000:1.0:1789302625.526872:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+63:19887:x1876219350195072:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:1.0:1789302625.526940:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302625.526977:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c0c0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcc0 expref: 63 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.527001:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f11d36700) 00010000:00010000:1.0:1789302625.527009:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302625.527446:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.527475:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2200/0x64fe43095184c0ff lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcd5 expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.527486:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.527565:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2200/0x64fe43095184c0ff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcd5 expref: 27 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.527584:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2200/0x64fe43095184c0ff lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcd5 expref: 27 pid: 5844 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302625.527934:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371050880:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302625.528152:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73af00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302625.534547:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371050880:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302625.534801:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302625.535695:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302625.536206:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350195200 00000100:00100000:1.0:1789302625.536221:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350195200 00010000:00010000:1.0:1789302625.536282:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2200/0x64fe43095184c0ff lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcd5 expref: 28 pid: 5844 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302625.536338:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 180 00000100:00100000:1.0:1789302625.536351:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:2347:x1876219350195200:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302625.536371:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302625.536396:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2200/0x64fe43095184c0ff lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcd5 expref: 28 pid: 5844 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.536411:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2200/0x64fe43095184c0ff lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcd5 expref: 28 pid: 5844 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.536443:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2200/0x64fe43095184c0ff lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fcd5 expref: 28 pid: 5844 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.536462:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f11d36700) 00010000:00010000:1.0:1789302625.536469:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.536481:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.536491:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.537603:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302625.537717:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:2347:x1876219350195200:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1362us (2023us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302625.537731:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 180 00010000:00010000:2.0:1789302625.541242:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.541263:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.541313:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c24f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.541330:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c24f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.541363:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c24f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302625.544190:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x26:0x0]: returning size 16384 00010000:00010000:2.0:1789302625.544217:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.544229:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.544238:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.544245:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.544251:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.544260:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.544293:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f11d36700) 00010000:00010000:2.0:1789302625.544297:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.544302:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.544310:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c248 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302625.544337:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006a2941b0 reply state 00000000faa6e225 transno 4294967715 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c24f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302625.544362:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302625.544768:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+64:19887:x1876219350195072:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 17892us (23277us total) trans 4294967715 rc 0/0 00000100:00100000:2.0:1789302625.544780:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 616 00000100:00100000:3.0:1789302625.552253:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302625.557354:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302625.557445:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350195456 00000100:00100000:2.0:1789302625.557458:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350195456 00000100:00100000:2.0:1789302625.557595:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00100000:2.0:1789302625.557608:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+64:19888:x1876219350195456:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302625.557640:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302625.557668:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c264 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd5a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.557757:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.557775:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.557791:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.557808:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.557815:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.557822:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558293:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c279 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558316:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c279 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558329:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c279 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302625.558381:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x26:0x0]: returning size 16384 00000004:00200000:2.0:1789302625.558386:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302625.558406:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c279 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558421:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558427:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558434:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558440:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.558445:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558452:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558465:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:2.0:1789302625.558481:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.558487:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558599:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c26b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558617:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558623:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558630:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558634:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.558638:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558644:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558652:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x843a253e (ffff960f02d72700) 00010000:00010000:2.0:1789302625.558655:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.558658:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558664:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c272 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x843a253e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558708:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c264 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd5a expref: 66 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558736:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbea00/0x64fe43095184c279 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd5a expref: 65 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.558744:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.558747:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302625.558749:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbea00, rc 0) 00000100:00100000:2.0:1789302625.563553:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+65:19888:x1876219350195456:12345-192.168.203.11@tcp:101:bash.0 Request processed in 5935us (6201us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302625.563567:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00100000:2.0:1789302625.570300:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350195584 00000100:00100000:2.0:1789302625.570310:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350195584 00000100:00100000:2.0:1789302625.570376:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 619 00000100:00100000:2.0:1789302625.570387:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+65:19888:x1876219350195584:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302625.570408:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302625.570430:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c280 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd61 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.570548:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.570562:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.570585:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572449:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572482:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572492:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572713:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572722:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572728:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572734:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.572739:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572746:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572758:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.572761:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.572766:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572772:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c287 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572784:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572795:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572801:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572806:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.572809:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572816:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572823:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff960f02d72700) 00010000:00010000:2.0:1789302625.572825:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.572828:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572832:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c28e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572876:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c280 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd61 expref: 66 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572885:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.572887:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.572890:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c280 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd61 expref: 66 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.572911:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf000, rc 0) 00000100:00100000:2.0:1789302625.574328:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+65:19888:x1876219350195584:12345-192.168.203.11@tcp:101:bash.0 Request processed in 3933us (4105us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302625.574350:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 619 00000100:00100000:1.0:1789302625.583984:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302625.598930:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350195712 00000100:00100000:3.0:1789302625.598947:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350195712 00000100:00100000:3.0:1789302625.599036:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 620 00000100:00100000:3.0:1789302625.599046:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19889:x1876219350195712:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302625.599070:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302625.599091:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c295 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd68 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.599156:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.599170:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.599181:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302625.599213:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302625.599216:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302625.599232:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.599253:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c295 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 29 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.599281:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 28 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.599290:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302625.599292:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302625.599296:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2cc00, rc 0) 00000100:00100000:3.0:1789302625.599357:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:19889:x1876219350195712:12345-192.168.203.11@tcp:101:rm.0 Request processed in 311us (15384us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302625.599369:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 620 00000100:00100000:3.0:1789302625.602145:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302625.610813:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350195968 00000100:00100000:3.0:1789302625.610831:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350195968 00000100:00100000:3.0:1789302625.610917:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 622 00000100:00100000:3.0:1789302625.610940:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19889:x1876219350195968:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302625.610969:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302625.611010:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c2bf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd76 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.611318:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.611349:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.611364:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.611385:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.611394:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.611401:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612486:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612638:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612648:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612655:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2de00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.612661:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612672:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612691:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.612695:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.612701:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612709:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c2c6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612726:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612751:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612757:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612762:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.612765:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612772:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612780:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff960f14168000) 00010000:00010000:2.0:1789302625.612783:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.612786:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612793:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c2cd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612861:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c2bf lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd76 expref: 30 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612875:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.612877:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.612881:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c2bf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd76 expref: 30 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.612904:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c400, rc 0) 00000100:00100000:2.0:1789302625.613273:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19889:x1876219350195968:12345-192.168.203.11@tcp:101:rm.0 Request processed in 2325us (2553us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302625.613284:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 622 00000100:00100000:1.0:1789302625.620220:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302625.648961:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302625.649188:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350196224 00000100:00100000:2.0:1789302625.649201:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350196224 00000100:00100000:2.0:1789302625.649293:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 624 00000100:00100000:2.0:1789302625.649306:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19890:x1876219350196224:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302625.649335:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302625.649363:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c2e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd84 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649452:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649470:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649484:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649541:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649549:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649557:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649645:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649654:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649661:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649668:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.649673:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649682:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649695:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.649699:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.649705:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649711:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c2f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649727:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649734:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649740:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649753:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.649757:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649763:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649772:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff960f02d72700) 00010000:00010000:2.0:1789302625.649774:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.649777:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.649783:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c2f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.650024:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c2e9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd84 expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.650036:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.650039:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.650043:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c2e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd84 expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.650054:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf000, rc 0) 00000100:00100000:2.0:1789302625.650140:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19890:x1876219350196224:12345-192.168.203.11@tcp:101:cp.0 Request processed in 836us (1195us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302625.650149:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 624 00000100:00100000:3.0:1789302625.655128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302625.655271:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350196352 00000100:00100000:2.0:1789302625.655307:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350196352 00000100:00100000:2.0:1789302625.655399:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 625 00000100:00100000:2.0:1789302625.655412:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19890:x1876219350196352:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302625.655443:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302625.655476:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c2fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd8b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655629:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655648:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655662:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655677:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655684:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655691:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655760:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655769:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655776:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655790:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.655797:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655915:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655944:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.655948:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.655955:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655961:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c305 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.655982:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656052:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656059:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656065:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302625.656068:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656075:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656097:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x88c75925 (ffff960f02d72700) 00010000:00010000:2.0:1789302625.656100:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.656103:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656110:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c30c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656150:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c2fe lrc: 1/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd8b expref: 30 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656160:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:2.0:1789302625.656163:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302625.656172:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184c2fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd8b expref: 30 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.656182:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbec00, rc 0) 00000100:00100000:2.0:1789302625.656381:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19890:x1876219350196352:12345-192.168.203.11@tcp:101:cp.0 Request processed in 970us (1266us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302625.656394:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 625 00000100:00100000:1.0:1789302625.660804:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302625.661427:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350196608 00000100:00100000:3.0:1789302625.661458:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350196608 00000100:00100000:3.0:1789302625.661553:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 626 00000100:00100000:3.0:1789302625.661565:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:19890:x1876219350196608:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302625.661594:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302625.661621:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c313 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd92 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.662600:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.662625:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.662639:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.662655:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c321 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.662662:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c321 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.662681:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c321 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302625.665353:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xe:0x0], index 0: 0 00010000:00010000:3.0:1789302625.666205:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666230:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666273:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666345:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666354:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666360:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666366:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2da00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302625.666371:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666378:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666391:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:3.0:1789302625.666394:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302625.666399:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.666406:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c31a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302625.666423:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009602db8e reply state 00000000b4dd14d0 transno 4294967716 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c321 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x26:0x0].0x88c75925 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302625.666432:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302625.666483:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c313 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x26:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 32 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.667260:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 31 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.667282:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302625.667285:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302625.667287:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d000, rc 0) 00000100:00100000:3.0:1789302625.673438:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:19890:x1876219350196608:12345-192.168.203.11@tcp:101:cp.0 Request processed in 11862us (12637us total) trans 4294967716 rc 0/0 00000100:00100000:3.0:1789302625.673453:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 626 00000100:00100000:1.0:1789302625.682789:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302625.682985:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350196736 00000100:00100000:3.0:1789302625.683004:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350196736 00000100:00100000:3.0:1789302625.683104:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 159 00000100:00100000:3.0:1789302625.683120:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+7:2348:x1876219350196736:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302625.683145:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:1.0:1789302625.683145:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789302625.683181:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c336 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fd99 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302625.683216:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xe:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302625.683244:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=462s v=(0 0) 00010000:00010000:3.0:1789302625.683249:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302625.683260:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c336 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd99 expref: 8 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.683292:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c336 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd99 expref: 8 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.683575:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:3.0:1789302625.683659:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350196736:12345-192.168.203.11@tcp:101:cp.0 Request processed in 543us (876us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302625.683666:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 159 00000100:00100000:1.0:1789302625.692812:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302625.707711:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350196992 00000100:00100000:2.0:1789302625.707729:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350196992 00010000:00010000:2.0:1789302625.707783:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c336 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd99 expref: 10 pid: 13439 timeout: 562 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302625.707826:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 181 00000100:00100000:2.0:1789302625.707836:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350196992:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302625.707869:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302625.707894:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c336 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd99 expref: 10 pid: 13439 timeout: 562 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.707909:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c336 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd99 expref: 10 pid: 13439 timeout: 562 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.707934:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302625.707939:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302625.707948:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000988aced1/0x64fe43095184c33d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda0 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.707959:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000988aced1/0x64fe43095184c33d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda0 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.707998:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000988aced1/0x64fe43095184c33d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda0 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302625.708011:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1876219371051136:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302625.708107:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66480 going to sleep for 11 seconds 00010000:00010000:1.0:1789302625.711595:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c336 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd99 expref: 10 pid: 13439 timeout: 562 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302625.711621:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371051008:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302625.711653:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000988aced1/0x64fe43095184c33d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda0 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.711668:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3200, rc 0) 00000100:00100000:1.0:1789302625.711786:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:19890:x1876219350196864:12345-192.168.203.11@tcp:101:cp.0 Request processed in 10176us (18983us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302625.711795:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 160 00000100:00100000:2.0:1789302625.725915:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302625.726014:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1876219371051136:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302625.726038:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302625.726096:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350196992:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 18263us (20068us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302625.726106:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 181 00000100:00100000:3.0:1789302625.732752:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350197248 00000100:00100000:3.0:1789302625.732768:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350197248 00000100:00100000:3.0:1789302625.732842:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 130 00000100:00100000:3.0:1789302625.732852:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:19890:x1876219350197248:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:3.0:1789302625.733204:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:19890:x1876219350197248:12345-192.168.203.11@tcp:35:cp.0 Request processed in 352us (7298us total) trans 4294967717 rc 0/0 00000100:00100000:3.0:1789302625.733211:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 130 00000100:00100000:2.0:1789302625.771722:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302625.782652:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350197760 00000100:00100000:2.0:1789302625.782666:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350197760 00000100:00100000:2.0:1789302625.782736:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 628 00000100:00100000:2.0:1789302625.782745:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+67:19893:x1876219350197760:12345-192.168.203.11@tcp:55:lfs.0 00010000:00010000:2.0:1789302625.782832:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.782850:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 31 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.782860:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c359 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda7 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.782868:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 5/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.782915:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c359 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda7 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.782930:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c359 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda7 expref: 67 pid: 5843 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302625.782948:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371051264:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302625.783053:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 31 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.783065:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 31 pid: 6919 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302625.783076:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371051392:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302625.783106:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66000 going to sleep for 11 seconds 00000100:00100000:3.0:1789302625.789627:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371051264:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302625.790799:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350197888 00000100:00100000:1.0:1789302625.790814:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350197888 00010000:00010000:1.0:1789302625.790865:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c359 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda7 expref: 68 pid: 5843 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302625.790975:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:1.0:1789302625.790987:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+68:2347:x1876219350197888:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302625.791004:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302625.791011:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c359 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda7 expref: 68 pid: 5843 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.791041:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c359 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda7 expref: 68 pid: 5843 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.791063:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c359 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fda7 expref: 68 pid: 5843 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.791082:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff960f14168100) 00010000:00010000:1.0:1789302625.791087:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.791097:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.791103:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302625.791171:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+67:2347:x1876219350197888:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 186us (465us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302625.791178:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:3.0:1789302625.792840:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371051392:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302625.792884:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302625.793781:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302625.799227:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 183 00000100:00100000:1.0:1789302625.799240:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:2345:x1876219350198016:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302625.799256:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302625.799273:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 32 pid: 6919 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.799288:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 32 pid: 6919 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.799330:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c328 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd92 expref: 32 pid: 6919 timeout: 562 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.799348:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff960f14168100) 00010000:00010000:1.0:1789302625.799353:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.799805:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.799814:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302625.799849:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302625.799975:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2345:x1876219350198016:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 735us (6196us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302625.799985:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 183 00010000:00010000:3.0:1789302625.800029:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.800046:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.802008:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.802047:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.802055:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.802061:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302625.802067:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.802075:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.802095:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x10:0x0].0x0 (ffff960f14168100) 00010000:00010000:3.0:1789302625.802098:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1789302625.802104:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.802111:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c367 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x10:0x0].0x0 bits 0x21/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302625.802499:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+67:19893:x1876219350197760:12345-192.168.203.11@tcp:55:lfs.0 Request processed in 19748us (20022us total) trans 4294967719 rc 0/0 00000100:00100000:3.0:1789302625.802532:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 628 00000100:00100000:3.0:1789302625.810916:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350198272 00000100:00100000:3.0:1789302625.810993:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350198272 00000100:00100000:3.0:1789302625.811117:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 131 00000100:00100000:3.0:1789302625.811141:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f07730c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+67:19893:x1876219350198272:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:3.0:1789302625.811770:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f07730c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+68:19893:x1876219350198272:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 630us (986us total) trans 4294967720 rc 0/0 00000100:00100000:3.0:1789302625.811791:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 131 00000100:00100000:2.0:1789302625.849028:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302625.858005:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350198528 00000100:00100000:2.0:1789302625.858018:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350198528 00000100:00100000:2.0:1789302625.858111:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 161 00000100:00100000:2.0:1789302625.858123:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350198528:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302625.858145:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302625.858170:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184c38a lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fdb5 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302625.858251:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371051520:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302625.858389:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66000 going to sleep for 11 seconds 00000100:00080000:3.0:1789302625.858620:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789302625.861313:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302625.861325:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb mtime: 0 -> 1789302625 00002000:00010000:2.0:1789302625.861328:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb atime: 0 -> 1789302625 00002000:00010000:2.0:1789302625.861330:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xe:0x0] updating lvb ctime: 0 -> 1789302625 00000100:00100000:2.0:1789302625.861364:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371051520:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302625.861414:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184c38a lrc: 1/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdb5 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.861445:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184c38a lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdb5 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302625.861463:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfc00, rc 0) 00000100:00100000:2.0:1789302625.861591:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350198528:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3470us (3765us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302625.861602:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 161 00000100:00100000:2.0:1789302625.864826:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302625.865184:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350198656 00000100:00100000:3.0:1789302625.865198:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350198656 00000100:00100000:3.0:1789302625.865287:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 630 00000100:00100000:3.0:1789302625.865299:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:19894:x1876219350198656:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302625.865325:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302625.865601:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c391 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fdbc expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.865698:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c398 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x26:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.865717:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c398 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.865731:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c398 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302625.865777:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x26:0x0]: returning size 16384 00000004:00200000:3.0:1789302625.865781:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302625.865802:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c398 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.865835:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c391 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdbc expref: 71 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.865859:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d000/0x64fe43095184c398 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x26:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdbc expref: 70 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.865891:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x26:0x0].0x0 (ffff960f02d72900) 00010000:00010000:3.0:1789302625.865894:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302625.865900:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d000, rc 0) 00000100:00100000:3.0:1789302625.866052:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:19894:x1876219350198656:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 753us (1238us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302625.866060:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 630 00000100:00100000:3.0:1789302625.869907:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350198784 00000100:00100000:3.0:1789302625.869931:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350198784 00000100:00100000:3.0:1789302625.870007:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 162 00000100:00100000:3.0:1789302625.870020:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350198784:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302625.870045:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302625.870069:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c39f lrc: 2/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fdc3 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302625.870147:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371051648:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302625.870276:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302625.873358:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371051648:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302625.873416:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c39f lrc: 1/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdc3 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.873446:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c39f lrc: 0/0,0 mode: --/PR res: [0x240000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdc3 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302625.873463:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:3.0:1789302625.873804:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350198784:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3784us (4209us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302625.873815:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 162 00000100:00100000:2.0:1789302625.888812:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302625.937038:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302626.960458:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302627.341301:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371051904:0@lo:400:kworker.0 00000100:00100000:2.0:1789302627.341315:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371051776:0@lo:400:kworker.0 00000100:00100000:1.0:1789302627.342408:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371052032:0@lo:400:kworker.0 00000100:00100000:2.0:1789302627.342770:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302627.342834:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371052288:0@lo:400:kworker.0 00000100:00100000:2.0:1789302627.342862:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302627.342979:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371052288 00000100:00100000:2.0:1789302627.342989:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371052288 00000100:00100000:2.0:1789302627.343055:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:2.0:1789302627.343064:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371052288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302627.343122:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371052288:12345-0@lo:400:kworker.0 Request processed in 60us (258us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302627.343128:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:2.0:1789302627.343170:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371052288:0@lo:400:kworker.0 00000100:00080000:2.0:1789302627.343203:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302627.345938:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302627.346005:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371052416:0@lo:400:kworker.0 00000100:00100000:3.0:1789302627.346058:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302627.354825:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302627.355197:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371052032 00000100:00100000:3.0:1789302627.355207:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371052032 00000100:00100000:3.0:1789302627.355303:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:3.0:1789302627.355316:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371052032:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302627.355401:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371052032:12345-0@lo:400:kworker.0 Request processed in 87us (585us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302627.355409:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00100000:3.0:1789302627.355461:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371052032:0@lo:400:kworker.0 00000100:00080000:2.0:1789302627.356287:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302627.597380:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350199424 00000100:00100000:1.0:1789302627.597387:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350199424 00000100:00100000:3.0:1789302627.597526:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350199936 00000100:00100000:3.0:1789302627.597531:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350199936 00000100:00100000:3.0:1789302627.597561:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 235 00000100:00100000:3.0:1789302627.597569:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350199552:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302627.597616:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350199552:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 48us (1511us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302627.597622:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 235 00000100:00100000:3.0:1789302627.597639:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 236 00000100:00100000:3.0:1789302627.597645:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350199936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302627.597667:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350199936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (1437us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302627.597684:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 236 00000100:00100000:2.0:1789302627.598104:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350199296 00000100:00100000:2.0:1789302627.598109:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350199296 00000100:00100000:2.0:1789302627.598146:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350199680 00000100:00100000:2.0:1789302627.598150:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350199680 00000100:00100000:2.0:1789302627.598175:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 273 00000100:00100000:2.0:1789302627.598183:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+66:2348:x1876219350199296:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302627.598216:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+66:2348:x1876219350199296:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (2166us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302627.598222:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 273 00000100:00100000:2.0:1789302627.598243:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 274 00000100:00100000:2.0:1789302627.598249:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:2348:x1876219350199680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302627.598285:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:2348:x1876219350199680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (2076us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302627.598292:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 274 00000100:00080000:2.0:1789302627.598443:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302627.598448:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302627.601121:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302627.852987:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371052544:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302627.853073:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302627.853585:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371052544:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302627.984842:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302628.367328:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371052672 00000100:00100000:3.0:1789302628.367336:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371052672 00000100:00100000:3.0:1789302628.367391:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:3.0:1789302628.367401:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371052672:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302628.367439:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 57868288 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302628.367493:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371052672:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302628.367520:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:1.0:1789302628.369121:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371052672:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302629.022053:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302630.028875:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302630.582744:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350200064 00000100:00100000:2.0:1789302630.582755:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350200064 00000100:00100000:2.0:1789302630.582833:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 632 00000100:00100000:2.0:1789302630.582844:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19886:x1876219350200064:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1789302630.582951:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19886:x1876219350200064:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 108us (322us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302630.582958:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 632 00000100:00100000:3.0:1789302630.584494:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302630.584624:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350200320 00000100:00100000:2.0:1789302630.584632:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350200320 00000100:00100000:2.0:1789302630.584680:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 132 00000100:00100000:2.0:1789302630.584689:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:19886:x1876219350200320:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302630.584948:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:19886:x1876219350200320:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 259us (458us total) trans 4294967721 rc 0/0 00000100:00100000:2.0:1789302630.584956:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 132 00000100:00100000:1.0:1789302630.669378:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302630.669478:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302630.669688:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350200448 00000100:00100000:3.0:1789302630.669699:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350200448 00002000:00100000:3.0:1789302630.669759:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xe:0x0] (0->157) req@ffff960f3f026680 x1876219350200448/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:206/0 lens 648/0 e 0 to 0 dl 1789302641 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302630.669770:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f026680 with x1876219350200448 ext(0->157) 00002000:00010000:3.0:1789302630.669776:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f026680 00000100:00100000:3.0:1789302630.669807:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 56 00000100:00100000:3.0:1789302630.669817:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350200448:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:3.0:1789302630.670355:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350200448:12345-192.168.203.11@tcp:4:cp.0 Request processed in 537us (874us total) trans 4294967327 rc 0/0 00000100:00100000:3.0:1789302630.670374:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 56 00002000:00100000:3.0:1789302630.670382:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0xe:0x0] (0->157) req@ffff960f3f026680 x1876219350200448/t4294967327(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:206/0 lens 648/448 e 0 to 0 dl 1789302641 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302630.670392:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f026680 with x1876219350200448 ext(0->157) 00002000:00010000:3.0:1789302630.670395:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f026680 00000100:00100000:2.0:1789302630.672029:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350200576 00000100:00100000:2.0:1789302630.672040:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350200576 00000100:00100000:2.0:1789302630.672111:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 163 00000100:00100000:2.0:1789302630.672120:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350200576:12345-192.168.203.11@tcp:16:kworker.0 00000001:00080000:1.0:1789302630.771727:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967327 is committed 00000100:00100000:3.0:1789302630.772036:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350200576:12345-192.168.203.11@tcp:16:kworker.0 Request processed in 99906us (102657us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302630.772050:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 163 00010000:00010000:2.0:1789302631.052777:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302631.523275:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967721 is committed 00010000:00010000:2.0:1789302632.078722:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1789302632.287851:0:16534:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 10d: Archive a file on the default archive id ========================================================== 08:30:30 (1789302630) 00000100:00100000:1.0:1789302632.462011:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371052928:0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462090:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302632.462136:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371053440:0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462161:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302632.462191:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302632.462197:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371053056:0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462229:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302632.462257:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371053440 00000100:00100000:1.0:1789302632.462265:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371053440 00000100:00100000:1.0:1789302632.462329:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:1.0:1789302632.462338:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371053440:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462388:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371053440:12345-0@lo:400:kworker.0 Request processed in 51us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.462395:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:1.0:1789302632.462418:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371052928 00000100:00100000:1.0:1789302632.462421:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371052928 00000100:00100000:1.0:1789302632.462445:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371053056 00000100:00100000:1.0:1789302632.462448:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371053056 00000100:00100000:1.0:1789302632.462465:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:1.0:1789302632.462472:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371052928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462526:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371052928:12345-0@lo:400:kworker.0 Request processed in 29us (411us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.462533:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:1.0:1789302632.462557:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1789302632.462564:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371053056:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462590:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371053056:12345-0@lo:400:kworker.0 Request processed in 27us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.462605:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1789302632.462642:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371053056:0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462652:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371052928:0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.462660:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371053440:0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.463715:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371052800 00000100:00100000:1.0:1789302632.463721:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371052800 00000100:00100000:1.0:1789302632.464940:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1789302632.464953:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371052800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302632.465024:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371052800:12345-0@lo:400:kworker.0 Request processed in 75us (2223us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.465032:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1789302632.598883:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:2.0:1789302632.601378:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302632.607226:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350200960 00000100:00100000:3.0:1789302632.607235:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350200960 00000100:00100000:3.0:1789302632.607375:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 634 00000100:00100000:3.0:1789302632.607385:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20496:x1876219350200960:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1789302632.607487:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20496:x1876219350200960:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 104us (587us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302632.607494:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 634 00000100:00100000:2.0:1789302632.611418:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.611548:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350201088 00000100:00100000:1.0:1789302632.611558:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350201088 00000100:00100000:1.0:1789302632.611621:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 24 00000100:00100000:1.0:1789302632.611631:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2346:x1876219350201088:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302632.611646:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302632.611671:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c3bb lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdca expref: 29 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.611689:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:1.0:1789302632.611694:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302632.611738:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:2346:x1876219350201088:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 109us (339us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.611746:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 24 00000100:00100000:2.0:1789302632.624453:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.624545:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350201216 00000100:00100000:1.0:1789302632.624556:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350201216 00000100:00100000:1.0:1789302632.624633:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 635 00000100:00100000:1.0:1789302632.624642:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+66:20497:x1876219350201216:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302632.624666:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302632.624685:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c3c9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fdd1 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.624745:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.624757:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.624768:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.624829:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.624837:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.624844:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625034:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625042:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625060:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625067:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302632.625070:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625079:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625091:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302632.625094:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.625099:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625105:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c3d0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625118:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625124:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625130:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625135:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302632.625138:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625154:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625162:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb54fab68 (ffff960f10d24b00) 00010000:00010000:1.0:1789302632.625164:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.625167:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625173:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3d7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625203:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c3c9 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdd1 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625213:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302632.625226:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.625230:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c3c9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdd1 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.625238:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99a00, rc 0) 00000100:00100000:1.0:1789302632.625308:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+66:20497:x1876219350201216:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 667us (863us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302632.625316:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 635 00000100:00080000:2.0:1789302632.625388:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302632.637446:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350201472 00000100:00100000:3.0:1789302632.637455:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350201472 00000100:00100000:3.0:1789302632.637530:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 636 00000100:00100000:3.0:1789302632.637541:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+66:20497:x1876219350201472:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:3.0:1789302632.637594:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302632.637621:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3800/0x64fe43095184c25d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd53 expref: 66 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.637639:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302632.637659:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302632.637864:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.637892:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 27 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.637900:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.637934:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 27 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.637947:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 27 pid: 6919 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302632.637961:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371053568:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302632.638047:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302632.641484:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.641716:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350201600 00000100:00100000:1.0:1789302632.641724:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350201600 00010000:00010000:1.0:1789302632.641760:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 28 pid: 6919 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302632.641810:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 184 00000100:00100000:1.0:1789302632.641819:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+28:2348:x1876219350201600:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302632.641836:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302632.641843:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 28 pid: 6919 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.641854:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 28 pid: 6919 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.641876:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c29c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fd68 expref: 28 pid: 6919 timeout: 569 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.641890:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302632.644258:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.644286:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.644295:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.644339:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302632.644412:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+27:2348:x1876219350201600:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 2589us (2924us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.644422:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 184 00010000:00010000:3.0:1789302632.644689:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.644700:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.644834:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c3e5 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.644845:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c3e5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.644853:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c3e5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302632.650739:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 16384 00010000:00010000:3.0:1789302632.650759:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.650771:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.650779:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.650784:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302632.650789:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.650796:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.650813:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302632.650816:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302632.650821:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.650827:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c3de lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302632.650850:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000fd7aa14e reply state 000000001b4c038a transno 4294967723 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c3e5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb54fab68 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302632.650859:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302632.650966:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+67:20497:x1876219350201472:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 13423us (13680us total) trans 4294967723 rc 0/0 00000100:00100000:3.0:1789302632.650984:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 636 00000100:00100000:2.0:1789302632.663724:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.666343:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350201728 00000100:00100000:1.0:1789302632.666375:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350201728 00000100:00100000:1.0:1789302632.666448:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 637 00000100:00100000:1.0:1789302632.666459:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+66:20498:x1876219350201728:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302632.666482:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302632.666540:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c3ec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fdd8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.666602:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3f3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.666616:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.666628:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302632.666664:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302632.666668:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302632.666684:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.666706:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c3ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdd8 expref: 68 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.666725:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3f3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdd8 expref: 67 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.666733:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302632.666736:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302632.666739:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99200, rc 0) 00000100:00100000:1.0:1789302632.666810:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+67:20498:x1876219350201728:12345-192.168.203.11@tcp:101:bash.0 Request processed in 352us (3095us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.666817:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 637 00000100:00100000:2.0:1789302632.669477:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.674541:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.674648:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350201984 00000100:00100000:1.0:1789302632.674658:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350201984 00000100:00100000:1.0:1789302632.674710:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 639 00000100:00100000:1.0:1789302632.674720:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+68:20498:x1876219350201984:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302632.674741:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302632.674761:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c416 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fde6 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674829:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674843:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674855:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674881:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674888:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674895:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674952:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674972:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674978:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674984:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302632.674989:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.674996:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675013:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff960f02dfd000) 00010000:00010000:1.0:1789302632.675016:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.675021:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675027:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c41d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675038:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675043:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675049:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675054:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302632.675064:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675071:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675078:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff960f10d24000) 00010000:00010000:1.0:1789302632.675080:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.675083:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675088:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c424 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675131:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c416 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fde6 expref: 69 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675142:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff960f02dfd000) 00010000:00010000:1.0:1789302632.675144:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.675148:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c416 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fde6 expref: 69 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.675158:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99a00, rc 0) 00000100:00100000:1.0:1789302632.675225:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+68:20498:x1876219350201984:12345-192.168.203.11@tcp:101:bash.0 Request processed in 498us (684us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302632.675233:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 639 00000100:00100000:2.0:1789302632.685666:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.691850:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.696126:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302632.696199:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350202368 00000100:00100000:3.0:1789302632.696209:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350202368 00000100:00100000:3.0:1789302632.696273:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00100000:3.0:1789302632.696283:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20499:x1876219350202368:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302632.696311:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302632.696331:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c455 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fdfb expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696394:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696408:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696419:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696433:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696439:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696445:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696494:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696537:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696544:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696551:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2ca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302632.696556:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696564:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696586:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff960f02dfd000) 00010000:00010000:3.0:1789302632.696590:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302632.696594:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696600:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c45c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696614:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696619:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696625:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696629:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302632.696639:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696645:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696652:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff960f0cdf2a00) 00010000:00010000:3.0:1789302632.696655:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302632.696658:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696662:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c463 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696692:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c455 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdfb expref: 30 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696701:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff960f02dfd000) 00010000:00010000:3.0:1789302632.696703:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302632.696706:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c455 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdfb expref: 30 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.696714:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:3.0:1789302632.696773:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20499:x1876219350202368:12345-192.168.203.11@tcp:101:rm.0 Request processed in 493us (655us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302632.696780:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00100000:2.0:1789302632.703584:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.703709:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350202496 00000100:00100000:1.0:1789302632.703719:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350202496 00000100:00100000:1.0:1789302632.703774:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 643 00000100:00100000:1.0:1789302632.703784:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20499:x1876219350202496:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302632.703818:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302632.703838:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c46a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe02 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.703898:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.703912:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.703922:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.703937:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.703944:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.703950:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.703999:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704006:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704011:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704017:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302632.704027:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704035:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704046:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff960f02dfd000) 00010000:00010000:1.0:1789302632.704049:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.704054:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704074:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c471 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704088:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704093:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704099:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704104:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302632.704106:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704112:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704119:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x7fd8c415 (ffff960f10d24000) 00010000:00010000:1.0:1789302632.704122:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.704132:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704138:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c478 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x27:0x0].0x7fd8c415 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704171:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c46a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe02 expref: 30 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704181:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff960f02dfd000) 00010000:00010000:1.0:1789302632.704183:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302632.704186:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c46a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe02 expref: 30 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.704194:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99a00, rc 0) 00000100:00100000:1.0:1789302632.704353:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20499:x1876219350202496:12345-192.168.203.11@tcp:101:rm.0 Request processed in 570us (788us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302632.704361:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 643 00000100:00100000:2.0:1789302632.723120:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.724986:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.725013:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.725030:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.725063:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.725089:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.725117:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.725142:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350202752 00000100:00100000:2.0:1789302632.725152:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350202752 00000100:00100000:2.0:1789302632.725205:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:2.0:1789302632.725230:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350202752:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302632.725327:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350203136 00000100:00100000:1.0:1789302632.725337:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350203136 00000100:00100000:2.0:1789302632.725594:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350202752:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 377us (528us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302632.725602:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:1.0:1789302632.725668:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350203264 00000100:00100000:1.0:1789302632.725678:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350203264 00000100:00100000:3.0:1789302632.725703:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:3.0:1789302632.725717:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350203392:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302632.725733:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:3.0:1789302632.725755:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350203392:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 42us (722us total) trans 0 rc 0/0 00000100:00080000:2.0:1789302632.725908:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302632.725912:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302632.725964:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350203008:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302632.726004:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350203008:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 47us (912us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302632.733539:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:3.0:1789302632.733891:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.736655:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:2.0:1789302632.741466:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.771874:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302632.772336:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350203904 00000100:00100000:3.0:1789302632.772345:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350203904 00000100:00100000:3.0:1789302632.772626:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 164 00000100:00100000:3.0:1789302632.772638:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350203904:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302632.772787:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302632.772810:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c4cc lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe1e expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302632.772837:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302632.772857:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=469s v=(0 0) 00010000:00010000:3.0:1789302632.772861:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302632.772871:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c4cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe1e expref: 9 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.772888:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c4cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe1e expref: 9 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.772901:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:3.0:1789302632.772963:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350203904:12345-192.168.203.11@tcp:101:cp.0 Request processed in 328us (1098us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302632.772970:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 164 00000100:00100000:2.0:1789302632.779536:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.791852:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302632.791999:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350204160 00000100:00100000:3.0:1789302632.792010:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350204160 00010000:00010000:3.0:1789302632.792048:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c4cc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe1e expref: 10 pid: 13439 timeout: 569 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302632.792110:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 185 00000100:00100000:3.0:1789302632.792130:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350204160:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302632.792147:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302632.792170:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c4cc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe1e expref: 10 pid: 13439 timeout: 569 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.792185:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c4cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe1e expref: 10 pid: 13439 timeout: 569 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.792209:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c4cc lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe1e expref: 10 pid: 13439 timeout: 569 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.792238:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302632.792243:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302632.792252:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 9 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.792480:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 9 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302632.792614:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 9 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302632.792629:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371053824:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302632.792720:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf840 going to sleep for 11 seconds 00000100:00100000:2.0:1789302632.798595:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302632.799046:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371053824:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302632.799069:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302632.799143:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350204160:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 7015us (7300us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302632.799153:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 185 00000100:00100000:2.0:1789302632.801143:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 133 00000100:00100000:2.0:1789302632.801155:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:20500:x1876219350204416:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302632.801682:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:20500:x1876219350204416:12345-192.168.203.11@tcp:35:cp.0 Request processed in 458us (3151us total) trans 4294967725 rc 0/0 00000100:00100000:2.0:1789302632.801692:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 133 00000100:00100000:2.0:1789302632.836952:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302632.845806:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350204672 00000100:00100000:2.0:1789302632.845816:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350204672 00000100:00100000:2.0:1789302632.845876:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 166 00000100:00100000:2.0:1789302632.845891:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350204672:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302632.845908:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302632.845928:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007223d374/0x64fe43095184c4f6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe33 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302632.845983:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371053952:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302632.846092:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66600 going to sleep for 11 seconds 00000100:00100000:3.0:1789302632.858167:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302632.858692:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350204800 00000100:00100000:1.0:1789302632.858702:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350204800 00000100:00100000:1.0:1789302632.859955:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:1.0:1789302632.859965:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:20503:x1876219350204800:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302632.859985:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302632.860022:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c4fd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe3a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.860101:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c504 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.860115:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.860126:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302632.860159:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x27:0x0]: returning size 16384 00000004:00200000:1.0:1789302632.860162:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302632.860179:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c504 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.860205:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c4fd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe3a expref: 71 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.860232:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99c00/0x64fe43095184c504 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe3a expref: 70 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.860240:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x27:0x0].0x0 (ffff960f02dfd000) 00010000:00010000:1.0:1789302632.860243:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302632.860247:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99c00, rc 0) 00000100:00100000:1.0:1789302632.862156:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:20503:x1876219350204800:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2181us (3988us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302632.862182:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:2.0:1789302632.870797:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302632.870883:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350204928 00000100:00100000:3.0:1789302632.870894:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350204928 00000100:00100000:3.0:1789302632.870949:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 167 00000100:00100000:3.0:1789302632.870958:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350204928:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302632.870976:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302632.870994:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c50b lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe41 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302632.872359:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371054080:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302632.872600:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66b40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302632.887101:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371054080:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302632.887149:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c50b lrc: 1/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe41 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.887187:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c50b lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe41 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302632.887202:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:1.0:1789302632.887287:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350204928:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 16329us (16498us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302632.887297:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 167 00000100:00100000:2.0:1789302632.895248:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302632.896903:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350205184 00000100:00100000:3.0:1789302632.896917:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350205184 00000100:00100000:2.0:1789302632.897228:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 649 00000100:00100000:2.0:1789302632.897239:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:20503:x1876219350205184:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:2.0:1789302632.897525:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].15, 136 off10232 00000100:00100000:2.0:1789302632.897611:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:20503:x1876219350205184:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 373us (2372us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302632.897619:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 649 00000100:00100000:3.0:1789302632.972946:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371054208:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302632.973025:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302632.974457:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371054208:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302633.118165:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302633.485635:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371054336:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302633.485711:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302633.486125:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371054336:0@lo:13:osp-pre-0-0.0 00000040:00080000:3.0:1789302633.682383:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302633.682392:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302633.682429:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 16 count 2 00000040:00080000:3.0:1789302633.682436:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302633.682696:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371054464:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302633.682786:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf300 going to sleep for 11 seconds 00000100:00100000:1.0:1789302633.686813:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371054464:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302633.690455:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302633.712722:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302633.715460:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:20513:x1876219350205568:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302633.715491:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302633.715526:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c512 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe48 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.715627:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c519 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.715641:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c519 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.715653:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c519 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302633.715664:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x11:0x0]: returning size 158 00010000:00010000:0.0:1789302633.715800:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c512 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe48 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.715817:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c519 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe48 expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.715825:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:0.0:1789302633.715827:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302633.715831:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3000, rc 0) 00000100:00100000:0.0:1789302633.716398:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:20513:x1876219350205568:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 941us (3681us total) trans 4294967726 rc 0/0 00000100:00100000:0.0:1789302633.716406:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 651 00000100:00100000:3.0:1789302633.723274:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302633.723459:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350205696 00000100:00100000:0.0:1789302633.723469:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350205696 00000100:00100000:0.0:1789302633.723568:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 168 00000100:00100000:0.0:1789302633.723586:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350205696:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302633.723627:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.723645:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 9 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.723652:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xe:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789302633.723656:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302633.723660:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 4/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.723691:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 9 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.723704:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 9 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302633.723718:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371054592:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302633.723805:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0600 going to sleep for 11 seconds 00000040:00080000:1.0:1789302633.730780:0:16583:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302633.730790:0:16583:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302633.730826:0:16583:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 16 count 2 00000040:00080000:1.0:1789302633.730831:0:16583:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302633.739375:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371054592:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302633.739412:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302633.750481:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302633.751447:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302633.751481:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350205952 00000100:00100000:1.0:1789302633.751491:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350205952 00000100:00100000:1.0:1789302633.751594:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 25 00000100:00100000:1.0:1789302633.751604:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2347:x1876219350205952:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302633.751620:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1789302633.751643:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350205824 00010000:00010000:1.0:1789302633.751644:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c519 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe48 expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302633.751648:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350205824 00010000:00010000:1.0:1789302633.751661:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:1.0:1789302633.751666:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00100000:0.0:1789302633.751680:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xe:0x0] (0->157) req@ffff960f3ef7ea00 x1876219350205824/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:209/0 lens 648/0 e 0 to 0 dl 1789302644 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302633.751690:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7ea00 with x1876219350205824 ext(0->157) 00010000:00010000:0.0:1789302633.751697:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 10 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302633.751713:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:2347:x1876219350205952:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 111us (269us total) trans 0 rc 0/0 00010000:00010000:0.0:1789302633.751717:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 10 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302633.751720:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 25 00002000:00010000:0.0:1789302633.751726:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ef7ea00 00000100:00100000:0.0:1789302633.751748:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 4 00000100:00100000:0.0:1789302633.751755:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350205824:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:0.0:1789302633.755227:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350205824:12345-192.168.203.11@tcp:4:cp.0 Request processed in 3454us (4735us total) trans 4294967328 rc 0/0 00000100:00100000:0.0:1789302633.755242:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 4 00002000:00100000:0.0:1789302633.755255:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xe:0x0] (0->157) req@ffff960f3ef7ea00 x1876219350205824/t4294967328(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:209/0 lens 648/448 e 0 to 0 dl 1789302644 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302633.755264:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7ea00 with x1876219350205824 ext(0->157) 00010000:00010000:0.0:1789302633.755274:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 12 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.755293:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 12 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302633.755301:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ef7ea00 00000100:00100000:2.0:1789302633.762323:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302633.762406:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350206080 00000100:00100000:3.0:1789302633.762417:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350206080 00010000:00010000:3.0:1789302633.762457:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 12 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302633.762517:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 186 00000100:00100000:3.0:1789302633.762528:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350206080:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302633.762542:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302633.762564:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 12 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762577:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 12 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762597:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000037e36333/0x64fe43095184c4d3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0xe:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe25 expref: 12 pid: 6695 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762609:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302633.762613:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302633.762619:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762628:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762665:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302633.762711:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350206080:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 184us (398us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302633.762718:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 186 00010000:00010000:3.0:1789302633.762742:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762750:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762771:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762779:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762786:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762793:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302633.762797:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762806:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762826:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.762896:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe43095184c527 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302633.762942:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350205696:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 39357us (39677us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302633.762950:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 168 00000100:00100000:2.0:1789302633.768112:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302633.768177:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350206336 00000100:00100000:3.0:1789302633.768187:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350206336 00000100:00100000:3.0:1789302633.768365:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 652 00000100:00100000:3.0:1789302633.768376:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:20513:x1876219350206336:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302633.768488:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:20513:x1876219350206336:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 113us (381us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302633.768496:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 652 00000100:00100000:3.0:1789302633.775092:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302633.775168:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350206464 00000100:00100000:2.0:1789302633.775178:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350206464 00000100:00100000:2.0:1789302633.775245:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 169 00000100:00100000:2.0:1789302633.775255:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350206464:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302633.775272:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302633.775299:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000cb7b2887/0x64fe43095184c52e lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe4f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302633.775339:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xe:0x0] initial LVB size: 158, mtime: 0x6aa69768, atime: 0x0, ctime: 0x6aa69768, blocks: 0x1 00010000:00010000:2.0:1789302633.775358:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=470s v=(0 0) 00010000:00010000:2.0:1789302633.775363:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302633.775373:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000cb7b2887/0x64fe43095184c52e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe4f expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.775390:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000cb7b2887/0x64fe43095184c52e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe4f expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.775402:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf400, rc 0) 00000100:00100000:2.0:1789302633.775460:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350206464:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 208us (378us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302633.775468:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 169 00000100:00100000:2.0:1789302633.780277:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302633.780363:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350206720 00000100:00100000:3.0:1789302633.780374:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350206720 00000100:00100000:3.0:1789302633.780434:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 653 00000100:00100000:3.0:1789302633.780446:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:20513:x1876219350206720:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302633.780579:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:20513:x1876219350206720:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 137us (317us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302633.780588:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 653 00000100:00100000:1.0:1789302633.798079:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302633.798112:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350206848 00000100:00100000:1.0:1789302633.798122:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350206848 00002000:00100000:1.0:1789302633.798164:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xe:0x0] (0->4095) req@ffff960f3ef7df80 x1876219350206848/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:209/0 lens 488/0 e 0 to 0 dl 1789302644 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302633.798176:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7df80 with x1876219350206848 ext(0->4095) 00002000:00010000:1.0:1789302633.798181:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ef7df80 00000100:00100000:1.0:1789302633.798203:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 57 00000100:00100000:1.0:1789302633.798223:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350206848:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302633.798451:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350206848:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 229us (381us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302633.798459:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 57 00002000:00100000:1.0:1789302633.798465:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0xe:0x0] (0->4095) req@ffff960f3ef7df80 x1876219350206848/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:209/0 lens 488/4536 e 0 to 0 dl 1789302644 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302633.798492:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7df80 with x1876219350206848 ext(0->4095) 00002000:00010000:1.0:1789302633.798496:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ef7df80 00000100:00100000:3.0:1789302633.802463:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302633.802532:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350206976 00000100:00100000:3.0:1789302633.802543:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350206976 00000100:00100000:3.0:1789302633.802593:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 654 00000100:00100000:3.0:1789302633.802604:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:20513:x1876219350206976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302633.802625:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302633.802648:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c535 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe56 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.802683:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.802697:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.802709:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.802810:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c535 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 35 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.802829:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.802838:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:3.0:1789302633.802841:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302633.802845:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2ca00, rc 0) 00000100:00100000:3.0:1789302633.802916:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:20513:x1876219350206976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 314us (466us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302633.802932:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 654 00000100:00100000:2.0:1789302633.824198:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302633.824246:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350207104 00000100:00100000:2.0:1789302633.824255:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350207104 00000100:00100000:2.0:1789302633.824304:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 655 00000100:00100000:2.0:1789302633.824315:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:20513:x1876219350207104:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302633.824347:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x11:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302633.824443:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:20513:x1876219350207104:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 129us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302633.824450:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 655 00000100:00100000:0.0:1789302633.841532:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302633.841643:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350207232 00000100:00100000:1.0:1789302633.841653:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350207232 00000100:00100000:1.0:1789302633.841711:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 170 00000100:00100000:1.0:1789302633.841722:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350207232:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302633.841776:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841793:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302633.841797:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xe:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302633.841803:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841817:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841846:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841855:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841864:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841884:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302633.841888:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841897:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841913:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302633.841921:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b6843458/0x64fe43095184c543 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xe:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302633.841989:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350207232:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 268us (487us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302633.841996:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 170 00000100:00100000:2.0:1789302633.850542:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302633.850863:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350207488 00000100:00100000:3.0:1789302633.850873:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350207488 00000100:00100000:3.0:1789302633.850929:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 656 00000100:00100000:3.0:1789302633.850941:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:20513:x1876219350207488:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302633.851020:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.851037:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.851046:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 4/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.851079:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302633.851093:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 34 pid: 5843 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302633.851107:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371054720:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302633.851446:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302633.862698:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdc700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371054720:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302633.862728:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302633.876860:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302633.877076:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350207616 00000100:00100000:0.0:1789302633.877088:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350207616 00010000:00010000:0.0:1789302633.877129:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 35 pid: 5843 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302633.877215:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 187 00000100:00100000:0.0:1789302633.877227:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:2348:x1876219350207616:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302633.877241:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302633.877247:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 35 pid: 5843 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.877261:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 35 pid: 5843 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.877282:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c53c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe56 expref: 35 pid: 5843 timeout: 570 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.877298:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:0.0:1789302633.877303:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.877313:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.877322:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302633.877355:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302633.877417:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2348:x1876219350207616:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 192us (567us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302633.877425:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 187 00010000:00010000:2.0:1789302633.880069:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880087:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880304:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880313:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880319:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880325:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302633.880330:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880337:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880353:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:2.0:1789302633.880356:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302633.880361:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302633.880367:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c54a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x11:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302633.880549:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:20513:x1876219350207488:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 29606us (30011us total) trans 4294967727 rc 0/0 00000100:00100000:2.0:1789302633.880568:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 656 00000040:00080000:1.0:1789302633.882054:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302633.882062:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302633.882548:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:1.0:1789302633.882558:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302633.892091:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302633.892195:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350207872 00000100:00100000:3.0:1789302633.892214:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350207872 00000100:00100000:3.0:1789302633.892275:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 134 00000100:00100000:3.0:1789302633.892286:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:20513:x1876219350207872:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302633.892552:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:20513:x1876219350207872:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 266us (469us total) trans 4294967728 rc 0/0 00000100:00100000:3.0:1789302633.892570:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 134 00010000:00010000:2.0:1789302634.125385:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302635.149032:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302635.696809:0:16635:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302635.696819:0:16635:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302635.696856:0:16635:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 16 count 2 00000040:00080000:3.0:1789302635.696862:0:16635:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302635.763298:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302635.763491:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350208128 00000100:00100000:3.0:1789302635.763517:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350208128 00000100:00080000:2.0:1789302635.763576:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302635.763584:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 657 00000100:00100000:3.0:1789302635.763595:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:20524:x1876219350208128:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302635.763614:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302635.763635:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c551 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe5d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302635.763730:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c558 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302635.763745:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c558 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302635.763758:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c558 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302635.763770:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x11:0x0]: returning size 158 00010000:00010000:3.0:1789302635.764360:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c551 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x27:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe5d expref: 73 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302635.764378:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c558 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe5d expref: 72 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302635.764397:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:3.0:1789302635.764400:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302635.764404:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2de00, rc 0) 00000100:00100000:3.0:1789302635.764466:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:20524:x1876219350208128:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 873us (1185us total) trans 4294967729 rc 0/0 00000100:00100000:3.0:1789302635.764473:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 657 00000100:00100000:1.0:1789302635.769542:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302635.769582:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302635.769611:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350208384 00000100:00100000:1.0:1789302635.769631:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350208384 00000100:00100000:1.0:1789302635.769697:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 658 00000100:00100000:1.0:1789302635.769706:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:20524:x1876219350208384:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:1.0:1789302635.769773:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769789:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769802:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769829:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769837:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769845:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769851:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302635.769856:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769863:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769874:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:1.0:1789302635.769877:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302635.769881:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302635.769898:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c566 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302635.769965:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:20524:x1876219350208384:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 259us (382us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302635.769973:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 658 00000100:00100000:3.0:1789302635.770212:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350208256 00000100:00100000:3.0:1789302635.770219:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350208256 00000100:00100000:3.0:1789302635.770261:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 26 00000100:00100000:3.0:1789302635.770269:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:2347:x1876219350208256:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302635.770283:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302635.770300:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c558 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe5d expref: 72 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302635.770311:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x11:0x0].0x0 (ffff960f02dfde00) 00010000:00010000:3.0:1789302635.770315:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302635.770344:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+71:2347:x1876219350208256:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 76us (811us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302635.770351:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 26 00000100:00100000:1.0:1789302635.789890:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302635.789976:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350208640 00000100:00100000:3.0:1789302635.789987:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350208640 00000100:00100000:3.0:1789302635.790047:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 135 00000100:00100000:3.0:1789302635.790056:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+71:20524:x1876219350208640:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:3.0:1789302635.790849:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:20524:x1876219350208640:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 791us (964us total) trans 4294967730 rc 0/0 00000100:00100000:3.0:1789302635.790866:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 135 00010000:00010000:2.0:1789302636.172851:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789302636.603393:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967328 is committed 00000001:00080000:2.0:1789302636.631527:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967726 is committed 00000001:00080000:2.0:1789302636.631551:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967727 is committed 00000001:00080000:2.0:1789302636.631559:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967730 is committed 00000001:00080000:2.0:1789302636.631566:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967728 is committed 00010000:00010000:2.0:1789302637.196800:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302637.587025:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371054976:0@lo:400:kworker.0 00000100:00100000:3.0:1789302637.587073:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371055104:0@lo:400:kworker.0 00000100:00100000:2.0:1789302637.587098:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302637.587131:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371054976 00000100:00100000:3.0:1789302637.587139:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371054976 00000100:00100000:2.0:1789302637.587165:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371055488:0@lo:400:kworker.0 00000100:00100000:3.0:1789302637.587199:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:3.0:1789302637.587208:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371054976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302637.587285:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371055232:0@lo:400:kworker.0 00000100:00100000:0.0:1789302637.587352:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371054848:0@lo:400:kworker.0 00000100:00100000:3.0:1789302637.587387:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371054976:12345-0@lo:400:kworker.0 Request processed in 179us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302637.587395:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:1.0:1789302637.588198:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302637.588248:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371055232 00000100:00100000:1.0:1789302637.588254:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371055232 00000100:00100000:1.0:1789302637.588300:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:1.0:1789302637.588308:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371055232:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302637.588481:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371055232:12345-0@lo:400:kworker.0 Request processed in 174us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302637.588488:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:1.0:1789302637.588605:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371055232:0@lo:400:kworker.0 00000100:00100000:0.0:1789302637.588719:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302637.588907:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371055360:0@lo:400:kworker.0 00000100:00100000:0.0:1789302637.588937:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302637.589025:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371054848 00000100:00100000:1.0:1789302637.589032:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371054848 00000100:00100000:1.0:1789302637.589068:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:1.0:1789302637.589074:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371054848:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302637.589116:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371054848:12345-0@lo:400:kworker.0 Request processed in 42us (397us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302637.589122:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00100000:1.0:1789302637.589149:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371055360 00000100:00100000:1.0:1789302637.589168:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371055360 00000100:00100000:0.0:1789302637.589295:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371054848:0@lo:400:kworker.0 00000100:00100000:1.0:1789302637.589463:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:1.0:1789302637.589471:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371055360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302637.589528:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371055360:12345-0@lo:400:kworker.0 Request processed in 58us (589us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302637.589534:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:2.0:1789302637.589918:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302637.589958:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371054976:0@lo:400:kworker.0 00000100:00080000:2.0:1789302637.589979:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302637.589984:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302637.589994:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371055488 00000100:00100000:2.0:1789302637.589998:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371055488 00000100:00100000:2.0:1789302637.590030:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1789302637.590037:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371055488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302637.590076:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371055488:12345-0@lo:400:kworker.0 Request processed in 39us (158us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302637.590082:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00080000:2.0:1789302637.590097:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302637.590126:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371055488:0@lo:400:kworker.0 00000100:00100000:3.0:1789302637.591116:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302637.602457:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371055104 00000100:00100000:0.0:1789302637.602474:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371055104 00000100:00100000:0.0:1789302637.602578:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:0.0:1789302637.602589:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371055104:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302637.602683:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371055104:12345-0@lo:400:kworker.0 Request processed in 97us (11570us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302637.602691:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00080000:0.0:1789302637.602718:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302637.602824:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371055360:0@lo:400:kworker.0 00000100:00100000:0.0:1789302637.602881:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371055104:0@lo:400:kworker.0 00000100:00100000:3.0:1789302637.841298:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302637.841406:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302637.841442:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302637.841462:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302637.841485:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302637.841523:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302637.841546:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302637.841572:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350209024 00000100:00100000:3.0:1789302637.841580:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350209024 00000100:00100000:1.0:1789302637.841626:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350208768 00000100:00100000:1.0:1789302637.841633:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350208768 00000100:00100000:3.0:1789302637.841634:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350209536 00000100:00100000:3.0:1789302637.841639:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350209536 00000100:00100000:3.0:1789302637.841660:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350209152 00000100:00100000:3.0:1789302637.841663:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350209152 00000100:00100000:1.0:1789302637.841674:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:3.0:1789302637.841680:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350209408 00000100:00100000:3.0:1789302637.841684:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350209408 00000100:00100000:1.0:1789302637.841688:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350208768:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302637.841703:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 247 00000100:00100000:3.0:1789302637.841711:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350209024:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302637.841756:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350208768:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 72us (465us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302637.841766:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:3.0:1789302637.841774:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350209024:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 47us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302637.841783:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 247 00000100:00100000:3.0:1789302637.841800:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 248 00000100:00100000:3.0:1789302637.841807:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350209536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302637.841947:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350209536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (370us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302637.841957:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 248 00000100:00100000:3.0:1789302637.841970:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:3.0:1789302637.841976:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350209152:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302637.842015:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350209152:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (490us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302637.842021:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:3.0:1789302637.842031:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 250 00000100:00100000:3.0:1789302637.842038:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350209408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302637.842057:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350209408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302637.842063:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 250 00000100:00100000:3.0:1789302637.842079:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350209280 00000100:00100000:3.0:1789302637.842083:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350209280 00000100:00100000:3.0:1789302637.842105:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350208896 00000100:00100000:3.0:1789302637.842108:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350208896 00000100:00100000:3.0:1789302637.842125:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:3.0:1789302637.842130:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2348:x1876219350209280:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302637.842166:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2348:x1876219350209280:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (723us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302637.842172:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:3.0:1789302637.842196:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:3.0:1789302637.842203:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:2348:x1876219350208896:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302637.842226:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:2348:x1876219350208896:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (741us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302637.842231:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00080000:3.0:1789302637.842258:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302637.842263:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302637.842265:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302638.099615:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302638.099647:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371055616:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302638.099741:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302638.099822:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371055616 00000100:00100000:0.0:1789302638.099832:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371055616 00000100:00100000:0.0:1789302638.099890:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 120 00000100:00100000:0.0:1789302638.099901:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371055616:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302638.099956:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 57868288 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302638.100006:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371055616:12345-0@lo:13:osp-pre-1-0.0 Request processed in 107us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302638.100013:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 120 00000100:00100000:0.0:1789302638.100071:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371055616:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302638.223214:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302638.615416:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371055744:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302638.615543:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302638.616016:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371055744 00000100:00100000:0.0:1789302638.616028:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371055744 00000100:00100000:0.0:1789302638.616097:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:0.0:1789302638.616109:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371055744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302638.616157:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302638.618069:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371055744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1948us (2510us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302638.618085:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1789302638.621554:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371055744:0@lo:13:osp-pre-0-0.0 00000040:00080000:3.0:1789302638.989795:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302638.989809:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302638.989853:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 16 count 2 00000040:00080000:3.0:1789302638.989861:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302639.245123:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302640.278369:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302641.292753:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302641.752345:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302641.754866:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350209664 00000100:00100000:3.0:1789302641.754882:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350209664 00000100:00100000:3.0:1789302641.755002:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 659 00000100:00100000:3.0:1789302641.755013:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20496:x1876219350209664:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1789302641.755179:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20496:x1876219350209664:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 167us (2844us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302641.755187:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 659 00000100:00080000:0.0:1789302641.756544:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302641.765074:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302641.765177:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350209920 00000100:00100000:0.0:1789302641.765187:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350209920 00000100:00100000:0.0:1789302641.765243:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 136 00000100:00100000:0.0:1789302641.765252:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:20496:x1876219350209920:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302641.765554:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:20496:x1876219350209920:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 301us (487us total) trans 4294967731 rc 0/0 00000100:00100000:0.0:1789302641.765562:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 136 00010000:00010000:2.0:1789302642.316972:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302642.710759:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371056000:0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.710780:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe32d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371056128:0@lo:400:kworker.0 00000100:00100000:2.0:1789302642.710861:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371055872:0@lo:400:kworker.0 00000100:00100000:1.0:1789302642.710864:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371056256:0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.720955:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302642.721023:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302642.721049:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371056512:0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.721092:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302642.722104:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371056128 00000100:00100000:3.0:1789302642.722114:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371056128 00000100:00100000:3.0:1789302642.722385:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789302642.722395:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371056128:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.722564:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371056128:12345-0@lo:400:kworker.0 Request processed in 171us (1615us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302642.722571:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789302642.722596:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371056512 00000100:00100000:3.0:1789302642.722600:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371056512 00000100:00100000:3.0:1789302642.722627:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:3.0:1789302642.722633:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371056512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.722666:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371056512:12345-0@lo:400:kworker.0 Request processed in 33us (1573us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302642.722672:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00080000:3.0:1789302642.722688:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302642.722693:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302642.722725:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371056512:0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.722737:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe32d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371056128:0@lo:400:kworker.0 00000100:00100000:0.0:1789302642.724203:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302642.724905:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371056000 00000100:00100000:3.0:1789302642.724914:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371056000 00000100:00100000:3.0:1789302642.724961:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789302642.724971:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371056000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.725048:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371056000:12345-0@lo:400:kworker.0 Request processed in 79us (850us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302642.725902:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789302642.725968:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371056000:0@lo:400:kworker.0 00000100:00100000:1.0:1789302642.730595:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302642.730642:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371056256 00000100:00100000:1.0:1789302642.730650:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371056256 00000100:00100000:1.0:1789302642.730722:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1789302642.730733:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371056256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302642.730788:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371056256:12345-0@lo:400:kworker.0 Request processed in 57us (199us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302642.730801:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00080000:1.0:1789302642.730822:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302642.730876:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371056256:0@lo:400:kworker.0 00000100:00100000:2.0:1789302642.733395:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302642.733534:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371056384:0@lo:400:kworker.0 00000100:00100000:2.0:1789302642.733580:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302642.733651:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371056384 00000100:00100000:3.0:1789302642.733660:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371056384 00000100:00100000:3.0:1789302642.733721:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1789302642.733731:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371056384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.733789:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371056384:12345-0@lo:400:kworker.0 Request processed in 59us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302642.733875:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1789302642.733922:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371056384:0@lo:400:kworker.0 00000100:00080000:1.0:1789302642.734104:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302642.737406:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371055872 00000100:00100000:0.0:1789302642.737422:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371055872 00000100:00100000:0.0:1789302642.737495:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:0.0:1789302642.737522:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371055872:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302642.737615:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371055872:12345-0@lo:400:kworker.0 Request processed in 112us (4226us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302642.737622:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1789302642.737847:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371055872:0@lo:400:kworker.0 00000100:00100000:3.0:1789302642.961586:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302642.961882:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302642.961919:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302642.961944:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302642.961967:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302642.961993:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302642.962018:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302642.962136:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210304 00000100:00100000:1.0:1789302642.962157:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210304 00000100:00100000:1.0:1789302642.962216:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210816 00000100:00100000:1.0:1789302642.962220:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210816 00000100:00100000:1.0:1789302642.962236:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210432 00000100:00100000:1.0:1789302642.962239:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210432 00000100:00100000:1.0:1789302642.962251:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210688 00000100:00100000:1.0:1789302642.962253:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210688 00000100:00100000:1.0:1789302642.962268:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:1.0:1789302642.962276:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350210304:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302642.962345:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350210304:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 69us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302642.962366:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:1.0:1789302642.962385:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:1.0:1789302642.962392:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350210816:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302642.962421:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350210816:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (476us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302642.962427:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:1.0:1789302642.962439:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 255 00000100:00100000:1.0:1789302642.962445:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f050a5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2345:x1876219350210432:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302642.962467:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f050a5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2345:x1876219350210432:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302642.962491:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 255 00000100:00100000:1.0:1789302642.962815:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 256 00000100:00100000:1.0:1789302642.962823:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350210688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302642.962851:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350210688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (832us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302642.962858:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 256 00000100:00080000:1.0:1789302642.962877:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302642.962881:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302642.963205:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210560 00000100:00100000:3.0:1789302642.963212:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210560 00000100:00100000:3.0:1789302642.963251:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210176 00000100:00100000:3.0:1789302642.963255:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210176 00000100:00100000:3.0:1789302642.963284:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 287 00000100:00100000:3.0:1789302642.963291:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:2345:x1876219350210560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302642.963340:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:2345:x1876219350210560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 45us (1416us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302642.963354:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 287 00000100:00100000:3.0:1789302642.963370:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:3.0:1789302642.963378:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:2345:x1876219350210176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302642.963407:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:2345:x1876219350210176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (1439us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302642.963413:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:0.0:1789302642.963683:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210048 00000100:00100000:0.0:1789302642.963688:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210048 00000100:00100000:0.0:1789302642.963720:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 349 00000100:00100000:0.0:1789302642.963727:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350210048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302642.963756:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350210048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (2178us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302642.963762:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 349 00000100:00080000:3.0:1789302642.966188:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302643.213701:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302643.213722:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371056640:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302643.213788:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302643.213847:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371056640 00000100:00100000:2.0:1789302643.213855:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371056640 00000100:00100000:2.0:1789302643.213899:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 122 00000100:00100000:2.0:1789302643.213907:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371056640:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302643.213940:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 57868288 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302643.213983:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371056640:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (194us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302643.213989:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 122 00000100:00100000:2.0:1789302643.214033:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371056640:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302643.343213:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1789302643.414082:0:16884:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 11a: Import a file ==================== 08:30:41 (1789302641) 00000100:00100000:3.0:1789302643.509134:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302643.509227:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350210944 00000100:00100000:1.0:1789302643.509236:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350210944 00000100:00100000:1.0:1789302643.509292:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 660 00000100:00100000:1.0:1789302643.509312:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:21113:x1876219350210944:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302643.509333:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302643.509353:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c56d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe64 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509419:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509435:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509447:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509464:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509490:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509497:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509974:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509983:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509991:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.509998:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302643.510002:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510011:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510024:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302643.510027:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302643.510032:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510038:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c574 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510049:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510060:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510066:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510071:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302643.510075:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510081:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510089:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff960f10d24100) 00010000:00010000:1.0:1789302643.510091:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302643.510094:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510100:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c57b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510135:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c56d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe64 expref: 70 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510158:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302643.510161:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302643.510165:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c56d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe64 expref: 70 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.510174:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99800, rc 0) 00000100:00100000:1.0:1789302643.510246:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:21113:x1876219350210944:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 936us (1121us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302643.510254:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 660 00000100:00100000:3.0:1789302643.534876:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302643.535005:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350211200 00000100:00100000:0.0:1789302643.535016:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350211200 00000100:00100000:0.0:1789302643.535066:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 661 00000100:00100000:0.0:1789302643.535076:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:21113:x1876219350211200:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:0.0:1789302643.535124:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302643.535157:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c3f3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fdd8 expref: 69 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.535174:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:0.0:1789302643.535179:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302643.535384:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.535398:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c432 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fded expref: 29 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.535418:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.535450:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c432 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fded expref: 29 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.535463:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c432 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fded expref: 29 pid: 5844 timeout: 580 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302643.535478:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371056768:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302643.535588:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0a9b1600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302643.537828:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371056768:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302643.537856:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302643.539029:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302643.539065:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350211328 00000100:00100000:3.0:1789302643.539073:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350211328 00010000:00010000:3.0:1789302643.539116:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c432 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fded expref: 30 pid: 5844 timeout: 580 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302643.539158:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 188 00000100:00100000:3.0:1789302643.539167:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:2347:x1876219350211328:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302643.539179:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302643.539185:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c432 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fded expref: 30 pid: 5844 timeout: 580 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302643.539197:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c432 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fded expref: 30 pid: 5844 timeout: 580 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302643.539275:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c432 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fded expref: 30 pid: 5844 timeout: 580 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302643.539491:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:3.0:1789302643.539498:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302643.539531:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302643.539538:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302643.539565:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302643.539618:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2347:x1876219350211328:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 451us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302643.539625:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 188 00000100:00080000:3.0:1789302643.539647:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302643.540054:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.540064:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.540088:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.540098:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.540107:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302643.541357:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 16384 00010000:00010000:1.0:1789302643.541371:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.541379:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.541385:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.541391:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302643.541395:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.541414:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.541428:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302643.541431:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302643.541437:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.541442:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c582 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302643.541458:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000798ed178 reply state 000000009d90aa07 transno 4294967732 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302643.541466:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302643.542033:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:21113:x1876219350211200:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 6953us (7160us total) trans 4294967732 rc 0/0 00000100:00100000:1.0:1789302643.542043:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 661 00010000:00010000:1.0:1789302643.544912:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.544930:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.544939:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.544946:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302643.544951:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.544992:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.545010:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff960f10d24100) 00010000:00010000:1.0:1789302643.545012:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302643.545019:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.545026:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c589 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302643.724914:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302643.724937:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371056896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302643.725002:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302643.725178:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371056896 00000100:00100000:2.0:1789302643.725187:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371056896 00000100:00100000:2.0:1789302643.725319:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:2.0:1789302643.725330:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371056896:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302643.725377:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302643.725585:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371056896:12345-0@lo:13:osp-pre-0-0.0 Request processed in 257us (583us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302643.725595:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00100000:3.0:1789302643.727397:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371056896:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302643.974725:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302643.974987:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350211456 00000100:00100000:1.0:1789302643.974997:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350211456 00000100:00100000:1.0:1789302643.975053:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00100000:1.0:1789302643.975064:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:21151:x1876219350211456:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:1.0:1789302643.975090:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302643.975112:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c590 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe6b expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.975185:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c597 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.975199:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c597 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.975211:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c597 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302643.975248:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302643.975264:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302643.975287:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c597 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.975311:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c590 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe6b expref: 71 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.975328:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c597 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe6b expref: 70 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.975337:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24100) 00010000:00010000:1.0:1789302643.975340:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302643.975343:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99a00, rc 0) 00000100:00100000:1.0:1789302643.975699:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:21151:x1876219350211456:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 635us (982us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302643.975707:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00100000:3.0:1789302643.981130:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302643.981230:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350211712 00000100:00100000:1.0:1789302643.981240:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350211712 00000100:00100000:1.0:1789302643.981301:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 663 00000100:00100000:1.0:1789302643.981312:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:21151:x1876219350211712:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:1.0:1789302643.981367:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302643.981407:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c597 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe6b expref: 70 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.981426:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24100) 00010000:00010000:1.0:1789302643.981436:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302643.981605:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:21151:x1876219350211712:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 294us (483us total) trans 0 rc -17/-17 00000100:00100000:1.0:1789302643.981628:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 663 00000100:00100000:3.0:1789302643.985615:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302643.985701:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350211840 00000100:00100000:0.0:1789302643.985711:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350211840 00000100:00100000:0.0:1789302643.985761:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 664 00000100:00100000:0.0:1789302643.985772:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+69:21151:x1876219350211840:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:0.0:1789302643.985793:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302643.985816:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c59e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe72 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.985882:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c5a5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.985897:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c5a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.985923:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c5a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302643.985959:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302643.985962:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302643.985980:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c5a5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.986007:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c59e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe72 expref: 71 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.986025:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c5a5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe72 expref: 70 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302643.986033:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302643.986036:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302643.986039:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7ec00, rc 0) 00000100:00100000:0.0:1789302643.986100:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:21151:x1876219350211840:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 330us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302643.986107:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 664 00000100:00100000:3.0:1789302643.992608:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302643.992709:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350211968 00000100:00100000:1.0:1789302643.992719:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350211968 00000100:00100000:1.0:1789302643.992778:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 665 00000100:00100000:1.0:1789302643.992788:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+70:21151:x1876219350211968:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:1.0:1789302643.992807:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302643.992827:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c5ac lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe79 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.992888:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.992903:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.992916:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.992939:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.992947:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.992953:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993037:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c5c1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993062:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c5c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993069:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c5c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302643.993102:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 16384 00000004:00200000:1.0:1789302643.993106:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302643.993123:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c5c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993134:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993151:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993158:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993164:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302643.993169:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993177:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993190:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302643.993193:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302643.993198:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993227:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5b3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993243:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993249:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993255:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993260:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302643.993263:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993269:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993277:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff960f10d24100) 00010000:00010000:1.0:1789302643.993279:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302643.993282:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993287:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5ba lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993316:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c5ac lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe79 expref: 72 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993329:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c5c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe79 expref: 71 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302643.993341:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302643.993344:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302643.993346:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99e00, rc 0) 00000100:00100000:1.0:1789302643.993408:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+71:21151:x1876219350211968:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 620us (809us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302643.993415:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 665 00000100:00100000:3.0:1789302644.087730:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302644.087837:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350212096 00000100:00100000:0.0:1789302644.087847:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350212096 00000100:00100000:0.0:1789302644.087906:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 666 00000100:00100000:0.0:1789302644.087916:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:21157:x1876219350212096:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302644.087945:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302644.087966:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c5c8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe80 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.088051:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.088064:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.088075:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302644.088109:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302644.088113:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302644.088132:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.088177:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c5c8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.088196:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 30 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.088205:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302644.088218:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302644.088222:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7e800, rc 0) 00000100:00100000:0.0:1789302644.088283:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:21157:x1876219350212096:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 368us (564us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302644.088290:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 666 00000100:00100000:3.0:1789302644.094342:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302644.094437:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350212352 00000100:00100000:1.0:1789302644.094447:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350212352 00000100:00100000:1.0:1789302644.094541:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 667 00000100:00100000:1.0:1789302644.094551:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+71:21157:x1876219350212352:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302644.094572:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302644.094575:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789302644.094602:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c5c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe79 expref: 71 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.094620:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302644.094631:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302644.094646:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c5d6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe87 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.094760:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.094772:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.094781:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.094854:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.094862:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.094869:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302644.095209:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302644.096021:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096033:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096043:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302644.096052:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 0 00010000:00010000:1.0:1789302644.096076:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096083:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096091:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096097:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.096100:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096121:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096130:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff960f10d24100) 00010000:00010000:1.0:1789302644.096132:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302644.096147:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096153:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c5dd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302644.096164:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005add9dff reply state 00000000e2eac786 transno 4294967733 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302644.096172:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302644.096203:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c5d6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 73 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096233:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 72 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.096240:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24500) 00010000:00010000:1.0:1789302644.096242:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302644.096244:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99200, rc 0) 00000100:00100000:1.0:1789302644.096316:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+73:21157:x1876219350212352:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1766us (1984us total) trans 4294967733 rc 0/0 00000100:00100000:1.0:1789302644.096323:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 667 00010000:00010000:2.0:1789302644.106069:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.106086:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.106093:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.106100:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302644.106105:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.106113:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.106131:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x4a4b4929 (ffff960f10d24400) 00010000:00010000:2.0:1789302644.106147:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302644.106154:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.106163:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c5e4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302644.108687:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302644.108822:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350212608 00000100:00100000:1.0:1789302644.108838:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350212608 00000100:00100000:1.0:1789302644.108896:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00100000:1.0:1789302644.108906:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:21157:x1876219350212608:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302644.108941:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302644.108967:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c5f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe8e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.109064:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.109079:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 73 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.109089:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.109122:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 73 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.109147:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 73 pid: 6919 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302644.109161:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371057024:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302644.109249:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43300 going to sleep for 11 seconds 00000100:00100000:3.0:1789302644.111889:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302644.112015:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350212736 00000100:00100000:2.0:1789302644.112025:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350212736 00010000:00010000:2.0:1789302644.112065:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 74 pid: 6919 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302644.112117:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 189 00000100:00100000:2.0:1789302644.112126:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:2347:x1876219350212736:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302644.112151:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302644.112159:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 74 pid: 6919 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.112171:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 74 pid: 6919 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.112193:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24500) 00010000:00010000:2.0:1789302644.112197:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.112206:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.112223:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.112242:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302644.112293:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:2347:x1876219350212736:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 168us (412us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302644.112300:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 189 00010000:00010000:1.0:1789302644.112620:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c5eb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe87 expref: 73 pid: 6919 timeout: 581 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302644.112636:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371057024:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302644.112662:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.112670:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.112675:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302644.112693:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 0 00010000:00010000:1.0:1789302644.113053:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113062:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113068:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113073:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.113077:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113084:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113093:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24500) 00010000:00010000:1.0:1789302644.113096:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302644.113099:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113104:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c600 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113177:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c5f9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe8e expref: 73 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113187:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff960f10d24700) 00010000:00010000:1.0:1789302644.113189:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302644.113192:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c5f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe8e expref: 73 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.113203:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2c00, rc 0) 00000100:00100000:1.0:1789302644.113895:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:21157:x1876219350212608:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4988us (5213us total) trans 4294967734 rc 301/301 00000100:00100000:1.0:1789302644.113904:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00100000:3.0:1789302644.126278:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302644.126370:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350212992 00000100:00100000:0.0:1789302644.126380:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350212992 00000100:00100000:0.0:1789302644.126464:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 137 00000100:00100000:0.0:1789302644.126475:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f07731050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:21157:x1876219350212992:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302644.126905:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f07731050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+73:21157:x1876219350212992:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 430us (634us total) trans 4294967735 rc 0/0 00000100:00100000:0.0:1789302644.126914:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 137 00000100:00100000:2.0:1789302644.141450:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302644.144888:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350213120 00000100:00100000:0.0:1789302644.144903:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350213120 00000100:00100000:1.0:1789302644.146345:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 669 00000100:00100000:1.0:1789302644.146358:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+73:21157:x1876219350213120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302644.146382:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302644.146419:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c60e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe95 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.146477:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c60e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe95 expref: 74 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.146491:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2c00/0x64fe43095184c60e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe95 expref: 74 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.146527:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24700) 00010000:00010000:1.0:1789302644.146543:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302644.146546:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2c00, rc 0) 00000100:00100000:1.0:1789302644.146614:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21157:x1876219350213120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 259us (5173us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302644.146622:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 669 00000100:00100000:2.0:1789302644.153184:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302644.153267:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350213376 00000100:00100000:3.0:1789302644.153277:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350213376 00000100:00100000:3.0:1789302644.153330:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 670 00000100:00100000:3.0:1789302644.153340:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21157:x1876219350213376:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1789302644.153401:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302644.153417:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302644.153428:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.153898:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.154036:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.154045:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.154052:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302644.154056:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.154064:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.154078:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24700) 00010000:00010000:2.0:1789302644.154081:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302644.154085:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302644.154091:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c615 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302644.154175:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:21157:x1876219350213376:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 835us (998us total) trans 4294967736 rc 0/0 00000100:00100000:2.0:1789302644.154183:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 670 00000100:00100000:2.0:1789302644.158394:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302644.158470:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350213632 00000100:00100000:3.0:1789302644.158480:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350213632 00000100:00100000:3.0:1789302644.158758:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 671 00000100:00100000:3.0:1789302644.158768:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:21157:x1876219350213632:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302644.159043:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302644.159059:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302644.159071:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302644.159147:0:6919:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967737 00000004:00010000:3.0:1789302644.159186:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000009f888d7 reply state 00000000aa5afc5b transno 4294967738 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302644.159217:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302644.161337:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21157:x1876219350213632:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 2560us (2940us total) trans 4294967738 rc 0/0 00000100:00100000:3.0:1789302644.161349:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 671 00000020:00080000:3.0:1789302644.161357:0:6919:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789302644.171312:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302644.171469:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350213888 00000100:00100000:3.0:1789302644.171480:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350213888 00000100:00100000:3.0:1789302644.171559:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 138 00000100:00100000:3.0:1789302644.171569:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21157:x1876219350213888:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302644.171807:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:21157:x1876219350213888:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 239us (504us total) trans 4294967739 rc 0/0 00000100:00100000:3.0:1789302644.171815:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 138 00010000:00010000:1.0:1789302644.172493:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.172573:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.172581:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.172587:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2de00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.172591:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.172599:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.172613:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24700) 00010000:00010000:1.0:1789302644.172616:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302644.172621:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.172628:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c61c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302644.214088:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302644.214191:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350214016 00000100:00100000:1.0:1789302644.214201:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350214016 00000100:00100000:1.0:1789302644.214276:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:1.0:1789302644.214301:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21160:x1876219350214016:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302644.214325:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302644.214344:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c623 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fe9c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.214408:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.214422:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.214433:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.214447:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.214454:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.214461:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214835:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c638 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214844:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c638 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214852:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c638 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302644.214878:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 16384 00000004:00200000:0.0:1789302644.214882:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302644.214897:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c638 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214908:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214916:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214922:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214928:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302644.214939:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214948:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214959:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302644.214963:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302644.214967:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214974:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c62a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214987:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214993:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.214999:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.215005:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302644.215008:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.215014:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.215021:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x4ba86b33 (ffff960f10d24500) 00010000:00010000:0.0:1789302644.215024:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302644.215033:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.215039:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c631 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x4ba86b33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.215071:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c623 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe9c expref: 80 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.215085:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f400/0x64fe43095184c638 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe9c expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.215092:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff960f02dfd600) 00010000:00010000:0.0:1789302644.215095:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302644.215097:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f400, rc 0) 00000100:00100000:0.0:1789302644.215173:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:21160:x1876219350214016:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 872us (1092us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302644.215180:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:2.0:1789302644.218847:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302644.219259:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350214272 00000100:00100000:1.0:1789302644.219269:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350214272 00000100:00100000:1.0:1789302644.219375:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 673 00000100:00100000:1.0:1789302644.219386:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:21160:x1876219350214272:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302644.219408:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302644.219428:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c63f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fea3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219584:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219599:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219610:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219625:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219631:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219637:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219746:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c654 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219754:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c654 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219762:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c654 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302644.219773:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 158 00010000:00010000:1.0:1789302644.219862:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219880:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219887:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219893:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.219898:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219906:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219918:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff960f02dfd600) 00010000:00010000:1.0:1789302644.219921:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302644.219926:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219932:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c646 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219944:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219949:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219955:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219960:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.219969:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219976:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219983:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x4a4b4929 (ffff960f10d24100) 00010000:00010000:1.0:1789302644.219986:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302644.219988:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.219994:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c64d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.220036:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c63f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fea3 expref: 82 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.220049:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99a00/0x64fe43095184c654 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fea3 expref: 81 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.220056:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24700) 00010000:00010000:1.0:1789302644.220058:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302644.220060:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99a00, rc 0) 00000100:00100000:1.0:1789302644.220124:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:21160:x1876219350214272:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 740us (1286us total) trans 4294967740 rc 0/0 00000100:00100000:1.0:1789302644.220132:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 673 00000100:00100000:3.0:1789302644.232462:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302644.232634:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350214400 00000100:00100000:0.0:1789302644.232644:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350214400 00000100:00100000:0.0:1789302644.232758:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 674 00000100:00100000:0.0:1789302644.232771:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:21160:x1876219350214400:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:0.0:1789302644.232836:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232853:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232865:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232908:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232916:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232924:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232931:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302644.232935:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232943:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232954:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24700) 00010000:00010000:0.0:1789302644.232957:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302644.232962:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302644.232969:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c662 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302644.233043:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:21160:x1876219350214400:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 274us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302644.233051:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 674 00000100:00100000:2.0:1789302644.256319:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302644.260253:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350214656 00000100:00100000:3.0:1789302644.260268:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350214656 00000100:00100000:0.0:1789302644.260458:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 139 00000100:00100000:0.0:1789302644.260470:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:21160:x1876219350214656:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:0.0:1789302644.260864:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:21160:x1876219350214656:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 395us (4555us total) trans 4294967741 rc 0/0 00000100:00100000:0.0:1789302644.260873:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 139 00010000:00010000:2.0:1789302644.364782:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302644.644634:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302644.644988:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350214912 00000100:00100000:1.0:1789302644.644997:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350214912 00000100:00100000:1.0:1789302644.645054:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 675 00000100:00100000:1.0:1789302644.645064:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:21180:x1876219350214912:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302644.645100:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302644.645120:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c669 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739feaa expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302644.645230:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x28:0x0]: returning size 16384 00010000:00010000:1.0:1789302644.645346:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c669 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739feaa expref: 84 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.645360:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302644.645363:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302644.645368:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c669 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739feaa expref: 84 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.645381:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc99200, rc 0) 00000100:00100000:1.0:1789302644.645440:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:21180:x1876219350214912:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 378us (815us total) trans 4294967742 rc 301/301 00000100:00100000:1.0:1789302644.645446:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 675 00000100:00100000:2.0:1789302644.654171:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302644.654284:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350215040 00000100:00100000:1.0:1789302644.654303:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350215040 00000100:00100000:1.0:1789302644.654359:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 676 00000100:00100000:1.0:1789302644.654368:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:21180:x1876219350215040:12345-192.168.203.11@tcp:34:lfs.0 00010000:00010000:1.0:1789302644.654438:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654455:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654466:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654493:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654524:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654531:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654610:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654618:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654624:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302644.654654:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2a:0x0]: returning size 158 00000004:00200000:1.0:1789302644.654658:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302644.654669:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654692:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654698:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654705:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654720:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.654725:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654732:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654743:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x0 (ffff960f02dfd600) 00010000:00010000:1.0:1789302644.654746:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302644.654751:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654756:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c677 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x28:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654768:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654773:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654778:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654782:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.654785:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654790:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654803:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x28:0x0].0x4a4b4929 (ffff960f10d24100) 00010000:00010000:1.0:1789302644.654805:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302644.654808:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654812:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c67e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x28:0x0].0x4a4b4929 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654824:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.654829:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.655106:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.655111:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302644.655113:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.655119:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.655126:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2a:0x0].0x0 (ffff960f10d24700) 00010000:00010000:1.0:1789302644.655128:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302644.655139:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302644.655144:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c685 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302644.655442:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:21180:x1876219350215040:12345-192.168.203.11@tcp:34:lfs.0 Request processed in 1073us (1278us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302644.655450:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 676 00000100:00100000:3.0:1789302644.666612:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302644.671979:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350215296 00000100:00100000:0.0:1789302644.671994:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350215296 00000100:00100000:0.0:1789302644.672069:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 140 00000100:00100000:0.0:1789302644.672078:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:21180:x1876219350215296:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:0.0:1789302644.673496:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:21180:x1876219350215296:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 1412us (6888us total) trans 4294967743 rc 0/0 00000100:00100000:0.0:1789302644.673522:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 140 00010000:00010000:2.0:1789302645.389049:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302646.427596:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302646.908102:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967743 is committed 00000001:00080000:2.0:1789302646.908135:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967731 is committed 00010000:00010000:2.0:1789302647.437097:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302647.821385:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789302647.821480:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371057280:0@lo:400:kworker.0 00000100:00100000:0.0:1789302647.821583:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302647.822193:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371057408:0@lo:400:kworker.0 00000100:00100000:3.0:1789302647.822229:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302647.822276:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371057280 00000100:00100000:3.0:1789302647.822284:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371057280 00000100:00100000:3.0:1789302647.822327:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371057408 00000100:00100000:3.0:1789302647.822330:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371057408 00000100:00100000:3.0:1789302647.822358:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:3.0:1789302647.822366:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371057280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302647.823895:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371057152:0@lo:400:kworker.0 00000100:00100000:1.0:1789302647.823955:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302647.824007:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371057152 00000100:00100000:1.0:1789302647.824014:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371057152 00000100:00100000:1.0:1789302647.824054:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1789302647.824061:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371057152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302647.824104:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371057152:12345-0@lo:400:kworker.0 Request processed in 43us (149us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302647.824122:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1789302647.824169:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371057152:0@lo:400:kworker.0 00000100:00100000:1.0:1789302647.824312:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:1.0:1789302647.824322:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371057408:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302647.824362:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371057408:12345-0@lo:400:kworker.0 Request processed in 42us (2132us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302647.824368:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00080000:1.0:1789302647.824409:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302647.831427:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371057664:0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.831495:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302647.831548:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302647.831555:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371057536:0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.831581:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302647.831613:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371057536 00000100:00100000:2.0:1789302647.831621:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371057536 00000100:00100000:2.0:1789302647.831674:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:2.0:1789302647.831686:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371057536:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.831736:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371057536:12345-0@lo:400:kworker.0 Request processed in 53us (153us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302647.831744:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00080000:2.0:1789302647.831783:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302647.831794:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371057664 00000100:00100000:2.0:1789302647.831798:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371057664 00000100:00100000:2.0:1789302647.831829:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:2.0:1789302647.831835:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371057664:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.831867:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371057664:12345-0@lo:400:kworker.0 Request processed in 32us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302647.831873:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00080000:2.0:1789302647.831890:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302647.831935:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371057536:0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.831945:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371057664:0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.831995:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371057408:0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.832004:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302647.832009:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371057792:0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.832035:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302647.832075:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371057792 00000100:00100000:2.0:1789302647.832079:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371057792 00000100:00100000:2.0:1789302647.832106:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:2.0:1789302647.832125:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371057792:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302647.832158:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371057792:12345-0@lo:400:kworker.0 Request processed in 46us (123us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302647.832164:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00080000:2.0:1789302647.832177:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302647.832329:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371057792:0@lo:400:kworker.0 00000100:00100000:3.0:1789302647.835995:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371057280:12345-0@lo:400:kworker.0 Request processed in 13620us (14403us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302647.836025:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:0.0:1789302647.842483:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371057280:0@lo:400:kworker.0 00000100:00100000:3.0:1789302648.076568:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302648.076599:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302648.076641:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302648.076671:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302648.076688:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302648.076705:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302648.076746:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302648.076775:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350215936 00000100:00100000:3.0:1789302648.076792:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350215936 00000100:00100000:3.0:1789302648.076849:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350215552 00000100:00100000:3.0:1789302648.076853:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350215552 00000100:00100000:3.0:1789302648.076876:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00100000:3.0:1789302648.076886:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2347:x1876219350215936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302648.076955:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2347:x1876219350215936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 70us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302648.076963:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00100000:3.0:1789302648.076982:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:3.0:1789302648.076989:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:2347:x1876219350215552:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302648.077019:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:2347:x1876219350215552:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302648.077026:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:0.0:1789302648.077094:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350215424 00000100:00100000:0.0:1789302648.077100:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350215424 00000100:00100000:0.0:1789302648.077143:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 351 00000100:00100000:0.0:1789302648.077158:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350215424:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302648.077204:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350215424:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 47us (644us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302648.077211:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 351 00000100:00100000:3.0:1789302648.077349:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350215680 00000100:00100000:3.0:1789302648.077354:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350215680 00000100:00100000:3.0:1789302648.077486:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350216192 00000100:00100000:3.0:1789302648.077490:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350216192 00000100:00100000:0.0:1789302648.083659:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350215808 00000100:00100000:0.0:1789302648.083673:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350215808 00000100:00100000:1.0:1789302648.084637:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350216064 00000100:00100000:1.0:1789302648.084645:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350216064 00000100:00100000:1.0:1789302648.084669:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00100000:1.0:1789302648.084679:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350215680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302648.084783:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350215680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 104us (8182us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302648.084791:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00100000:1.0:1789302648.084810:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 260 00000100:00100000:1.0:1789302648.084817:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350216192:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302648.084842:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350216192:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (8171us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302648.084847:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 260 00000100:00100000:1.0:1789302648.084859:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:1.0:1789302648.084865:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350215808:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302648.084887:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350215808:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (8181us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302648.084893:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:1.0:1789302648.084903:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 262 00000100:00100000:1.0:1789302648.084909:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350216064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302648.084930:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350216064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (8185us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302648.084935:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 262 00000100:00080000:2.0:1789302648.085173:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302648.085178:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302648.085190:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:3.0:1789302648.204753:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302648.204762:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302648.205059:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 15 count 1 00000040:00080000:3.0:1789302648.205067:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302648.333822:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371057920:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302648.333899:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302648.334392:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371057920 00000100:00100000:1.0:1789302648.334401:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371057920 00000100:00100000:1.0:1789302648.334457:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 124 00000100:00100000:1.0:1789302648.334467:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371057920:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302648.334620:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 57868288 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302648.334689:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371057920:12345-0@lo:13:osp-pre-1-0.0 Request processed in 223us (788us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302648.334695:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 124 00000100:00100000:2.0:1789302648.335010:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371057920:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302648.461017:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302648.845701:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302648.845723:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371058048:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302648.845796:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302648.845882:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371058048 00000100:00100000:1.0:1789302648.845891:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371058048 00000100:00100000:1.0:1789302648.845942:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:1.0:1789302648.845951:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371058048:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302648.845985:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302648.846030:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371058048:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302648.846037:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:0.0:1789302648.846165:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371058048:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302649.487583:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302650.511733:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302651.533398:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302652.561264:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302652.599493:0:17083:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 11b: Import a deleted file using its FID ========================================================== 08:30:51 (1789302651) 00000100:00100000:3.0:1789302652.797012:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302652.797345:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350216576 00000100:00100000:1.0:1789302652.797366:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350216576 00000100:00100000:1.0:1789302652.797453:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 677 00000100:00100000:1.0:1789302652.797467:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:21801:x1876219350216576:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302652.797487:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302652.797546:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c68c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739feb8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797612:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797625:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797635:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797649:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797668:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797675:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797842:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797849:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797854:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797859:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.797863:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797869:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797880:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302652.797884:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.797888:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797894:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c693 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797903:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797914:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797919:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797923:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.797925:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797930:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797936:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff960f10d24100) 00010000:00010000:1.0:1789302652.797938:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.797941:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797946:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c69a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797976:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c68c lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739feb8 expref: 73 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797985:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302652.797988:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.797991:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c68c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739feb8 expref: 73 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.797998:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fc98600, rc 0) 00000100:00100000:1.0:1789302652.798069:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:21801:x1876219350216576:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 608us (1071us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302652.798087:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 677 00000100:00080000:0.0:1789302652.798380:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302652.800477:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302652.800578:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350216448 00000100:00100000:2.0:1789302652.800591:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350216448 00000100:00080000:0.0:1789302652.800684:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302652.800685:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 678 00000100:00100000:2.0:1789302652.800703:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:21800:x1876219350216448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302652.800741:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302652.800767:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c6a1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739feb1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.800856:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.800873:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.800887:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.800904:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.800911:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.800918:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.801244:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c6b6 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.801255:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c6b6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.801284:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c6b6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302652.801297:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00000100:00100000:3.0:1789302652.801728:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302652.801796:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350216832 00000100:00100000:1.0:1789302652.801809:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350216832 00000100:00100000:1.0:1789302652.801886:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 679 00000100:00100000:1.0:1789302652.801898:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+72:21801:x1876219350216832:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:1.0:1789302652.801956:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302652.801982:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c5a5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe72 expref: 72 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.802004:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302652.802009:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302652.802128:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.802139:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.802208:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.802216:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302652.802221:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.802231:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.802243:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:2.0:1789302652.802247:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302652.802252:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.802255:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.802273:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.802283:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.802319:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.802333:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302652.802366:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371058176:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302652.802466:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43cc0 going to sleep for 11 seconds 00010000:00010000:2.0:1789302652.803111:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c6a8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.803136:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.803206:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302652.803928:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371058176:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302652.803954:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.804006:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.804014:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302652.804018:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.804027:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.804044:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff960f11d36d00) 00010000:00010000:2.0:1789302652.804047:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302652.804053:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.804059:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184c6af lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.804969:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184c6a1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739feb1 expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.804993:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c6b6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739feb1 expref: 31 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.805002:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302652.805005:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302652.805008:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe000, rc 0) 00000100:00100000:2.0:1789302652.805244:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21800:x1876219350216448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4543us (4774us total) trans 4294967744 rc 0/0 00000100:00100000:2.0:1789302652.805267:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 678 00000100:00100000:3.0:1789302652.806758:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302652.806986:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350216960 00000100:00100000:0.0:1789302652.806998:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350216960 00010000:00010000:0.0:1789302652.807044:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302652.807087:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 190 00000100:00100000:0.0:1789302652.807099:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2348:x1876219350216960:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302652.807114:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302652.807122:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.807137:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.807173:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c5cf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fe80 expref: 31 pid: 5845 timeout: 589 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.807191:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302652.807196:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.807206:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.807214:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.807268:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302652.807319:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.807347:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.807382:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.807397:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.807408:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302652.807478:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:2348:x1876219350216960:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 379us (724us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302652.807488:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 190 00000004:00200000:1.0:1789302652.808421:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 16384 00010000:00010000:1.0:1789302652.808434:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.808443:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.808452:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.808458:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.808463:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.808480:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.808495:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302652.808498:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.808545:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.808553:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6c4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302652.808579:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ec082387 reply state 00000000d8d647b6 transno 4294967745 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302652.808590:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302652.808683:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+73:21801:x1876219350216832:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 6785us (6953us total) trans 4294967745 rc 0/0 00000100:00100000:1.0:1789302652.808693:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 679 00000100:00100000:3.0:1789302652.811461:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302652.811651:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217088 00000100:00100000:2.0:1789302652.811664:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217088 00000100:00100000:2.0:1789302652.811769:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 680 00000100:00100000:2.0:1789302652.811781:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:21800:x1876219350217088:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302652.811912:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:21800:x1876219350217088:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 134us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.811920:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 680 00000100:00100000:3.0:1789302652.815348:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302652.816306:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217216 00000100:00100000:2.0:1789302652.816321:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217216 00000100:00100000:3.0:1789302652.817116:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 27 00000100:00100000:3.0:1789302652.817134:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:2347:x1876219350217216:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302652.817161:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302652.817195:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe000/0x64fe43095184c6b6 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739feb1 expref: 30 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.817219:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302652.817227:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302652.817355:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:2347:x1876219350217216:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 227us (2020us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302652.817364:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 27 00000100:00100000:0.0:1789302652.819041:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302652.819346:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217344 00000100:00100000:3.0:1789302652.819357:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217344 00000100:00100000:3.0:1789302652.819418:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 681 00000100:00100000:3.0:1789302652.819430:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+73:21802:x1876219350217344:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302652.819460:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302652.819485:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c6d2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739febf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.819621:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c6d9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.819638:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c6d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.819652:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c6d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302652.819697:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302652.819701:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302652.819725:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c6d9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.819753:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c6d2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739febf expref: 75 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.819773:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c6d9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739febf expref: 74 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.819782:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:3.0:1789302652.819785:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302652.819789:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2ca00, rc 0) 00000100:00100000:3.0:1789302652.819851:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21802:x1876219350217344:12345-192.168.203.11@tcp:101:bash.0 Request processed in 424us (820us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302652.819859:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 681 00010000:00010000:2.0:1789302652.820471:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.820483:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.820492:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.820498:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc98600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302652.820610:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.820622:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.820636:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff960f10d24100) 00010000:00010000:2.0:1789302652.820640:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302652.820658:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.820665:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98600/0x64fe43095184c6cb lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302652.824945:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302652.825062:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217472 00000100:00100000:1.0:1789302652.825082:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217472 00000100:00100000:1.0:1789302652.825292:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 682 00000100:00100000:1.0:1789302652.825370:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+73:21802:x1876219350217472:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302652.825398:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302652.825425:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6e0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fec6 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825536:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825575:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825589:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825605:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825612:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825619:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825739:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c6f5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825746:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c6f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825754:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c6f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302652.825786:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 16384 00000004:00200000:1.0:1789302652.825790:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302652.825813:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c6f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825828:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825845:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825852:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825859:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.825864:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825872:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825886:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:1.0:1789302652.825889:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.825894:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825900:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99e00/0x64fe43095184c6e7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825913:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825919:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825925:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825929:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fc99800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.825936:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825942:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825948:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff960f10d24b00) 00010000:00010000:1.0:1789302652.825949:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.825952:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825957:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99800/0x64fe43095184c6ee lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.825990:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc99200/0x64fe43095184c6e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fec6 expref: 75 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.826004:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e800/0x64fe43095184c6f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fec6 expref: 74 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.826012:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:1.0:1789302652.826014:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302652.826016:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7e800, rc 0) 00000100:00100000:1.0:1789302652.826100:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21802:x1876219350217472:12345-192.168.203.11@tcp:101:bash.0 Request processed in 798us (1169us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302652.826106:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 682 00000100:00100000:0.0:1789302652.829013:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302652.829093:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217600 00000100:00100000:3.0:1789302652.829122:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217600 00000100:00100000:3.0:1789302652.829189:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 683 00000100:00100000:3.0:1789302652.829371:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21802:x1876219350217600:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302652.829401:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302652.829430:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c6fc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fecd expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.829565:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.829581:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.829594:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.829877:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.829900:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.829913:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830815:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830827:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830834:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830856:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.830860:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830868:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830883:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.830886:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.830891:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830899:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c703 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830914:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830920:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830927:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830933:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2da00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.830936:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830942:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.830960:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302652.830963:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.830966:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.831807:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c70a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.831871:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c6fc lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fecd expref: 75 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.831884:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.831887:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.831892:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c6fc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fecd expref: 75 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.831902:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d400, rc 0) 00000100:00100000:3.0:1789302652.832933:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21802:x1876219350217600:12345-192.168.203.11@tcp:101:bash.0 Request processed in 2674us (2981us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302652.832944:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 683 00000100:00100000:0.0:1789302652.841615:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302652.842368:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217728 00000100:00100000:2.0:1789302652.842383:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217728 00000100:00100000:2.0:1789302652.842469:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 684 00000100:00100000:2.0:1789302652.842482:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+29:21803:x1876219350217728:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302652.842827:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302652.842856:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c711 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fed4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.842936:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.842953:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.843360:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302652.843408:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302652.843412:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302652.843435:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.843466:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c711 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 31 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.844297:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.844311:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:2.0:1789302652.844315:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302652.844319:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf000, rc 0) 00000100:00100000:3.0:1789302652.845874:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:21803:x1876219350217728:12345-192.168.203.11@tcp:101:rm.0 Request processed in 3379us (4257us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302652.845888:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 684 00000100:00100000:2.0:1789302652.854950:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302652.856303:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217856 00000100:00100000:1.0:1789302652.856320:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217856 00000100:00100000:1.0:1789302652.856403:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 685 00000100:00100000:1.0:1789302652.856415:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:21803:x1876219350217856:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302652.856461:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302652.856489:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c71f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fedb expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.856780:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.856892:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.856905:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.856923:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.856931:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.856939:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.857054:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eac00/0x64fe43095184c734 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.857064:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eac00/0x64fe43095184c734 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.857071:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eac00/0x64fe43095184c734 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302652.857120:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 16384 00000004:00200000:1.0:1789302652.857130:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302652.858276:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eac00/0x64fe43095184c734 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858300:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858307:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858316:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858322:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.858326:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858336:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858351:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:1.0:1789302652.858355:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.858361:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858368:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c726 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858383:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.858389:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859609:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859616:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eae00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.859621:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859632:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859647:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xbc29f2cb (ffff960f10d24b00) 00010000:00010000:1.0:1789302652.859650:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.859655:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859662:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c72d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xbc29f2cb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859732:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184c71f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fedb expref: 32 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859750:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eac00/0x64fe43095184c734 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fedb expref: 31 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.859757:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:1.0:1789302652.859759:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302652.859761:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eac00, rc 0) 00000100:00100000:1.0:1789302652.859860:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21803:x1876219350217856:12345-192.168.203.11@tcp:101:rm.0 Request processed in 3445us (4922us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302652.859882:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 685 00000100:00100000:1.0:1789302652.862890:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302652.862971:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350217984 00000100:00100000:3.0:1789302652.862984:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350217984 00000100:00100000:3.0:1789302652.863055:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 686 00000100:00100000:3.0:1789302652.863067:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21803:x1876219350217984:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302652.863122:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302652.863214:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c73b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fee2 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863313:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863331:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863345:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863361:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863368:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863376:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863685:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863694:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863702:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863708:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2da00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.863720:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863728:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863744:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.863747:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.863752:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863758:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c742 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863772:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863779:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863785:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863790:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.863793:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863799:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863806:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302652.863809:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.863818:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863824:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c749 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863868:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c73b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fee2 expref: 32 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863879:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.863882:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.863885:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c73b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fee2 expref: 32 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.863894:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d400, rc 0) 00000100:00100000:3.0:1789302652.864030:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21803:x1876219350217984:12345-192.168.203.11@tcp:101:rm.0 Request processed in 964us (1156us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302652.864038:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 686 00000100:00100000:0.0:1789302652.875763:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302652.875879:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350218112 00000100:00100000:3.0:1789302652.875893:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350218112 00000100:00100000:3.0:1789302652.875988:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 687 00000100:00100000:3.0:1789302652.876000:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21803:x1876219350218112:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302652.876025:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302652.876049:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c750 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fee9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876152:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876192:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876206:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876224:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876232:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876239:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876308:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876316:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876323:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876330:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2de00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.876335:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876344:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876357:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.876366:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.876372:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876379:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c757 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876409:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876415:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876422:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876428:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.876431:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876438:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876447:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302652.876449:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.876452:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876459:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c75e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876521:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c750 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fee9 expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876541:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.876543:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.876548:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c750 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fee9 expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.876558:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2cc00, rc 0) 00000100:00100000:3.0:1789302652.876754:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21803:x1876219350218112:12345-192.168.203.11@tcp:101:rm.0 Request processed in 756us (1006us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302652.876762:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 687 00000100:00100000:3.0:1789302652.890989:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302652.891117:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350218240 00000100:00100000:0.0:1789302652.891131:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350218240 00000100:00100000:0.0:1789302652.891249:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 688 00000100:00100000:0.0:1789302652.891263:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21804:x1876219350218240:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302652.891290:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302652.891318:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c765 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fef0 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891403:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891420:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891432:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891448:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891455:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891479:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891740:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891749:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891756:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891763:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302652.891768:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891777:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891793:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:0.0:1789302652.891796:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302652.891802:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891808:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c76c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891824:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891829:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891842:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891849:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302652.891851:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891857:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891865:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f09ae1100) 00010000:00010000:0.0:1789302652.891867:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302652.891869:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891875:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c773 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891920:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c765 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fef0 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891929:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:0.0:1789302652.891932:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302652.891935:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c765 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fef0 expref: 32 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302652.891943:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2800, rc 0) 00000100:00100000:0.0:1789302652.892031:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21804:x1876219350218240:12345-192.168.203.11@tcp:101:cp.0 Request processed in 771us (1057us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302652.892048:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 688 00000100:00100000:0.0:1789302652.895965:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302652.896252:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350218368 00000100:00100000:3.0:1789302652.896266:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350218368 00000100:00100000:3.0:1789302652.896344:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 689 00000100:00100000:3.0:1789302652.896357:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21804:x1876219350218368:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302652.896384:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302652.896410:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c77a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fef7 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896488:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896529:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896541:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896593:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896601:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896608:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896676:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896683:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896690:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896696:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.896700:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896708:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896720:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.896723:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.896729:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896737:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c781 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896760:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896767:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896774:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896779:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2de00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.896782:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896789:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896796:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302652.896799:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.896802:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896808:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c788 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896848:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c77a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fef7 expref: 32 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896860:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.896863:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.896873:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c77a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fef7 expref: 32 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.896882:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2cc00, rc 0) 00000100:00100000:3.0:1789302652.896963:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21804:x1876219350218368:12345-192.168.203.11@tcp:101:cp.0 Request processed in 609us (1013us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302652.896971:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 689 00000100:00100000:2.0:1789302652.901482:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302652.901857:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350218624 00000100:00100000:3.0:1789302652.901870:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350218624 00000100:00100000:3.0:1789302652.901971:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 690 00000100:00100000:3.0:1789302652.901982:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21804:x1876219350218624:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302652.902019:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302652.902046:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c78f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fefe expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.902439:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.902458:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.902472:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.902488:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.902495:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.902710:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302652.906474:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0xf:0x0], index 0: 0 00010000:00010000:3.0:1789302652.907019:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907040:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907055:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907110:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907119:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907126:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907132:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302652.907138:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907818:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907839:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302652.907842:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302652.907849:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907878:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c796 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302652.907899:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000021ce05b6 reply state 0000000084b3a0e3 transno 4294967746 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302652.907909:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302652.907963:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c78f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 34 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907980:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 33 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.907988:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302652.907990:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302652.907993:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2cc00, rc 0) 00000100:00100000:3.0:1789302652.908102:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21804:x1876219350218624:12345-192.168.203.11@tcp:101:cp.0 Request processed in 6121us (6635us total) trans 4294967746 rc 0/0 00000100:00100000:3.0:1789302652.908113:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 690 00010000:00010000:1.0:1789302652.911887:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.911910:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.911918:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.911924:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2da00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302652.911930:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.911940:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.911961:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f0cdf2600) 00010000:00010000:1.0:1789302652.911965:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302652.911970:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.911977:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c79d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302652.913380:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302652.914283:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350218752 00000100:00100000:3.0:1789302652.914297:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350218752 00000100:00100000:3.0:1789302652.914420:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 171 00000100:00100000:3.0:1789302652.914432:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350218752:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302652.914454:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302652.914480:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff05 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302652.914565:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302652.914602:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=489s v=(0 0) 00010000:00010000:3.0:1789302652.914608:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302652.914619:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.914640:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302652.914657:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2de00, rc 0) 00000100:00100000:3.0:1789302652.914743:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350218752:12345-192.168.203.11@tcp:101:cp.0 Request processed in 313us (1374us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302652.914751:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 171 00000100:00100000:1.0:1789302652.918103:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302652.918451:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350218880 00000100:00100000:2.0:1789302652.918464:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350218880 00000100:00100000:2.0:1789302652.918549:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 172 00000100:00100000:2.0:1789302652.918568:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:21804:x1876219350218880:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302652.918599:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302652.918624:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff0c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.918667:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.918677:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302652.918682:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302652.918688:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 4/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.918743:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.918780:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 10 pid: 6696 timeout: 589 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302652.918799:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371058304:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302652.918914:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75900 going to sleep for 11 seconds 00000100:00080000:0.0:1789302652.919853:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302652.926746:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371058304:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302652.926800:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 3/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302652.926826:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf800, rc 0) 00000100:00100000:2.0:1789302652.926902:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:21804:x1876219350218880:12345-192.168.203.11@tcp:101:cp.0 Request processed in 8338us (8813us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.926913:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 172 00000100:00100000:2.0:1789302652.932017:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302652.934734:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350219008 00000100:00100000:1.0:1789302652.934758:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350219008 00010000:00010000:1.0:1789302652.934834:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 10 pid: 6696 timeout: 589 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302652.934990:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 191 00000100:00100000:1.0:1789302652.935003:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350219008:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302652.935021:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302652.935051:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 10 pid: 6696 timeout: 589 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.935067:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 10 pid: 6696 timeout: 589 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.935107:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e3704795/0x64fe43095184c7b2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff05 expref: 10 pid: 6696 timeout: 589 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.935130:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302652.935135:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302652.935157:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.935169:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302652.935224:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302652.935239:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371058432:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302652.935369:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302652.939343:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302652.939591:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371058432:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302652.939621:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302652.939694:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350219008:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4693us (7690us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.939706:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 191 00000100:00100000:3.0:1789302652.941816:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350219264 00000100:00100000:3.0:1789302652.941834:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350219264 00000100:00100000:0.0:1789302652.942049:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 141 00000100:00100000:0.0:1789302652.942062:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:21804:x1876219350219264:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302652.944167:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21804:x1876219350219264:12345-192.168.203.11@tcp:35:cp.0 Request processed in 2083us (4812us total) trans 4294967747 rc 0/0 00000100:00100000:0.0:1789302652.944184:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 141 00000100:00100000:0.0:1789302652.945468:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371058688:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.945674:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371058560:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.945720:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302652.945767:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371059072:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.945821:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371058816:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.945856:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371058560 00000100:00100000:2.0:1789302652.945865:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371058560 00000100:00100000:2.0:1789302652.945919:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:2.0:1789302652.945928:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371058560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.945989:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371058560:12345-0@lo:400:kworker.0 Request processed in 61us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.945996:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:3.0:1789302652.946103:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371058944:0@lo:400:kworker.0 00000100:00100000:3.0:1789302652.946155:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302652.946350:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371058944 00000100:00100000:2.0:1789302652.946355:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371058944 00000100:00100000:2.0:1789302652.946415:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:2.0:1789302652.946423:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371058944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.946469:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371058944:12345-0@lo:400:kworker.0 Request processed in 48us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.946475:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00100000:2.0:1789302652.946554:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371058944:0@lo:400:kworker.0 00000100:00100000:0.0:1789302652.947308:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302652.947374:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371059200:0@lo:400:kworker.0 00000100:00100000:0.0:1789302652.947418:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302652.947533:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371059200 00000100:00100000:2.0:1789302652.947542:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371059200 00000100:00100000:2.0:1789302652.947604:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1789302652.947613:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371059200:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.947666:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371059200:12345-0@lo:400:kworker.0 Request processed in 55us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.947672:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1789302652.947706:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371059200:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.947728:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302652.947784:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302652.947816:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371058560:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.947838:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371058688 00000100:00100000:2.0:1789302652.947842:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371058688 00000100:00100000:2.0:1789302652.947866:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371058816 00000100:00100000:2.0:1789302652.947870:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371058816 00000100:00100000:2.0:1789302652.947883:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371059072 00000100:00100000:2.0:1789302652.947885:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371059072 00000100:00100000:2.0:1789302652.947897:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:2.0:1789302652.947904:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371058688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.947942:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371058688:12345-0@lo:400:kworker.0 Request processed in 38us (2365us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.947948:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00100000:2.0:1789302652.947960:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:2.0:1789302652.947966:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371058816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.947997:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371058816:12345-0@lo:400:kworker.0 Request processed in 31us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.948003:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:2.0:1789302652.948013:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:2.0:1789302652.948018:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371059072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.948049:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371059072:12345-0@lo:400:kworker.0 Request processed in 31us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302652.948054:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:2.0:1789302652.948091:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371058688:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.948114:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371058816:0@lo:400:kworker.0 00000100:00100000:2.0:1789302652.948134:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371059072:0@lo:400:kworker.0 00000100:00080000:3.0:1789302652.948467:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302652.948474:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302652.948476:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302653.006873:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302653.007003:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350219392 00000100:00100000:2.0:1789302653.007016:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350219392 00000100:00100000:2.0:1789302653.007138:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 691 00000100:00100000:2.0:1789302653.007276:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21807:x1876219350219392:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302653.007305:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302653.007330:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe200/0x64fe43095184c7c0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff13 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007415:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007432:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007446:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007462:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007469:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007475:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007683:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c7d5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007693:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c7d5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007721:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c7d5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302653.007827:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 158 00000004:00200000:2.0:1789302653.007833:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302653.007857:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c7d5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007872:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c7d5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007882:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007889:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007897:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007903:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302653.007909:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007918:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007931:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:2.0:1789302653.007935:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302653.007941:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007957:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184c7c7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007974:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007980:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007986:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.007992:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302653.007995:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.008002:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.008010:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f11d36d00) 00010000:00010000:2.0:1789302653.008013:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302653.008016:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.008021:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ce lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.008059:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe200/0x64fe43095184c7c0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff13 expref: 76 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.008097:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c7d5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff13 expref: 75 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.008105:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302653.008107:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302653.008110:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302653.008798:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:21807:x1876219350219392:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1524us (1936us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302653.008809:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 691 00000100:00100000:1.0:1789302653.013082:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.013407:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350219520 00000100:00100000:3.0:1789302653.013421:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350219520 00000100:00100000:3.0:1789302653.013524:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 173 00000100:00100000:3.0:1789302653.013537:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350219520:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302653.013560:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302653.013808:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c7dc lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff1a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302653.013886:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371059328:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302653.014043:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1900 going to sleep for 11 seconds 00000100:00080000:0.0:1789302653.014287:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:2.0:1789302653.016653:0:6697:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302653.016664:0:6697:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb mtime: 0 -> 1789302652 00002000:00010000:2.0:1789302653.016667:0:6697:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb atime: 0 -> 1789302652 00002000:00010000:2.0:1789302653.016669:0:6697:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0xf:0x0] updating lvb ctime: 0 -> 1789302652 00000100:00100000:2.0:1789302653.016703:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371059328:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302653.016815:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c7dc lrc: 1/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff1a expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.016844:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c7dc lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff1a expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.016860:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d400, rc 0) 00000100:00100000:2.0:1789302653.016984:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350219520:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3449us (3923us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302653.016995:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 173 00000100:00100000:1.0:1789302653.020755:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302653.021400:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350219648 00000100:00100000:2.0:1789302653.021416:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350219648 00000100:00100000:2.0:1789302653.021743:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:2.0:1789302653.021758:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:21807:x1876219350219648:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302653.021788:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302653.021815:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe200/0x64fe43095184c7e3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff21 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.021905:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ea lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.021922:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.021937:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302653.021987:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 16384 00000004:00200000:2.0:1789302653.021991:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302653.022013:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.022044:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe200/0x64fe43095184c7e3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff21 expref: 77 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.022067:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff21 expref: 76 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.022087:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:2.0:1789302653.022090:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302653.022115:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfe00, rc 0) 00000100:00100000:2.0:1789302653.022214:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:21807:x1876219350219648:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 460us (1483us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302653.022221:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:1.0:1789302653.025800:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.025930:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350219776 00000100:00100000:3.0:1789302653.025944:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350219776 00000100:00100000:3.0:1789302653.026038:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 174 00000100:00100000:3.0:1789302653.026049:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350219776:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302653.026085:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302653.026109:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c7f1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff28 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302653.026291:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371059456:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302653.026404:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1b40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302653.028626:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371059456:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302653.028678:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c7f1 lrc: 1/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff28 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.028708:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f6c72544/0x64fe43095184c7f1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff28 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.028726:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:3.0:1789302653.028918:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350219776:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2869us (3198us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302653.028928:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 174 00000100:00100000:1.0:1789302653.033060:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302653.033329:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350220032 00000100:00100000:0.0:1789302653.033342:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350220032 00000100:00100000:0.0:1789302653.033442:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 693 00000100:00100000:0.0:1789302653.033454:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:21807:x1876219350220032:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:0.0:1789302653.033929:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].16, 136 off10368 00000100:00100000:0.0:1789302653.034084:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:21807:x1876219350220032:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 621us (1028us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302653.034092:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 693 00000100:00100000:1.0:1789302653.195975:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.196015:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.196058:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.196111:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350220288 00000100:00100000:1.0:1789302653.196123:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350220288 00000100:00100000:3.0:1789302653.196251:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350220160 00000100:00100000:3.0:1789302653.196255:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350220160 00000100:00100000:3.0:1789302653.196313:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 353 00000100:00100000:3.0:1789302653.196324:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350220160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302653.196411:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350220160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 88us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302653.196420:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 353 00000100:00100000:1.0:1789302653.196965:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350220416 00000100:00100000:1.0:1789302653.196970:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350220416 00000100:00100000:1.0:1789302653.196998:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 265 00000100:00100000:1.0:1789302653.197006:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2345:x1876219350220288:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302653.197051:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2345:x1876219350220288:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 46us (1092us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.197058:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 265 00000100:00100000:1.0:1789302653.197087:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 266 00000100:00100000:1.0:1789302653.197094:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350220416:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302653.197118:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350220416:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (1059us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.197124:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 266 00000100:00080000:3.0:1789302653.197430:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302653.213461:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.213542:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302653.213686:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350220544 00000100:00100000:0.0:1789302653.213699:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350220544 00002000:00100000:0.0:1789302653.213773:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (0->157) req@ffff960f095ac000 x1876219350220544/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:229/0 lens 648/0 e 0 to 0 dl 1789302664 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302653.213800:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f095ac000 with x1876219350220544 ext(0->157) 00002000:00010000:0.0:1789302653.213809:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f095ac000 00000100:00100000:0.0:1789302653.213843:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 58 00000100:00100000:0.0:1789302653.213854:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350220544:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:3.0:1789302653.214244:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350220672 00000100:00100000:3.0:1789302653.214251:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350220672 00000100:00100000:3.0:1789302653.214291:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 175 00000100:00100000:3.0:1789302653.214299:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350220672:12345-192.168.203.11@tcp:16:kworker.0 00000100:00100000:3.0:1789302653.219833:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350220672:12345-192.168.203.11@tcp:16:kworker.0 Request processed in 5409us (6262us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302653.219853:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 175 00000100:00100000:0.0:1789302653.224037:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350220544:12345-192.168.203.11@tcp:4:cp.0 Request processed in 10167us (10476us total) trans 4294967328 rc 0/0 00000100:00100000:0.0:1789302653.224056:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 58 00002000:00100000:0.0:1789302653.224080:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (0->157) req@ffff960f095ac000 x1876219350220544/t4294967328(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:229/0 lens 648/448 e 0 to 0 dl 1789302664 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302653.224092:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f095ac000 with x1876219350220544 ext(0->157) 00002000:00010000:0.0:1789302653.224099:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f095ac000 00000040:00080000:3.0:1789302653.324855:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302653.324869:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302653.324915:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 17 count 2 00000040:00080000:3.0:1789302653.324924:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302653.325279:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371059584:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302653.325470:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302653.332269:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371059584:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302653.344987:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.345133:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350220800 00000100:00100000:3.0:1789302653.346277:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350220800 00000100:00100000:3.0:1789302653.346374:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 694 00000100:00100000:3.0:1789302653.346387:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21817:x1876219350220800:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302653.347305:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x12:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302653.347456:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21817:x1876219350220800:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 1071us (2485us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302653.347466:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 694 00000100:00100000:2.0:1789302653.360212:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.360405:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350221056 00000100:00100000:3.0:1789302653.360421:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350221056 00000100:00100000:3.0:1789302653.360494:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 695 00000100:00100000:3.0:1789302653.360530:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21817:x1876219350221056:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302653.360559:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302653.360587:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c7f8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff2f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.360725:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c7ff lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.360742:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c7ff lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.360756:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c7ff lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302653.360769:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 158 00010000:00010000:3.0:1789302653.360942:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c7f8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff2f expref: 37 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.360968:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c7ff lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff2f expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.361007:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302653.361011:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302653.361016:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d400, rc 0) 00000100:00100000:3.0:1789302653.361102:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21817:x1876219350221056:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 597us (906us total) trans 4294967748 rc 0/0 00000100:00100000:3.0:1789302653.361111:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 695 00000100:00100000:0.0:1789302653.364616:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302653.364730:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350221312 00000100:00100000:2.0:1789302653.364743:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350221312 00000100:00100000:2.0:1789302653.364811:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 176 00000100:00100000:2.0:1789302653.364823:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350221312:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302653.364884:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.364911:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.364922:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0xf:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789302653.364927:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302653.364933:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 4/1,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.364978:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.364995:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 11 pid: 6697 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302653.365013:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371059712:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302653.365138:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75900 going to sleep for 11 seconds 00000100:00100000:3.0:1789302653.365378:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.365473:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350221184 00000100:00100000:3.0:1789302653.365478:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350221184 00000100:00100000:3.0:1789302653.365550:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 28 00000100:00100000:3.0:1789302653.365558:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350221184:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302653.365570:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302653.365597:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c7ff lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff2f expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.365617:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302653.365623:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302653.365783:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:2348:x1876219350221184:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 225us (405us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302653.365791:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 28 00000100:00100000:2.0:1789302653.368679:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371059712:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302653.368719:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/1,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302653.371398:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302653.371773:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350221440 00000100:00100000:2.0:1789302653.371787:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350221440 00010000:00010000:2.0:1789302653.371841:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 12 pid: 6697 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302653.371883:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 192 00000100:00100000:2.0:1789302653.371894:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350221440:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302653.371913:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302653.371942:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 12 pid: 6697 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.371957:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 12 pid: 6697 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.371989:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003192f3c1/0x64fe43095184c7b9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff0c expref: 12 pid: 6697 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.372010:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302653.372015:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302653.372023:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.372056:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 4/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302653.372106:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302653.372223:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372234:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372276:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372286:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372295:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372304:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302653.372309:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372320:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372335:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.372343:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184c80d lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302653.372440:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350221440:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 546us (1053us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302653.372449:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 192 00000100:00100000:3.0:1789302653.381562:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350221312:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 16723us (16945us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302653.381586:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 176 00000100:00100000:3.0:1789302653.392590:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.392724:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350221696 00000100:00100000:1.0:1789302653.392740:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350221696 00000100:00100000:1.0:1789302653.392821:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 696 00000100:00100000:1.0:1789302653.392835:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:21817:x1876219350221696:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302653.393000:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:21817:x1876219350221696:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 168us (424us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.393009:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 696 00000100:00100000:2.0:1789302653.398630:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.398713:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350221824 00000100:00100000:1.0:1789302653.398727:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350221824 00000100:00100000:1.0:1789302653.398800:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:1.0:1789302653.398812:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350221824:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302653.398836:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302653.398862:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff36 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302653.398898:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0xf:0x0] initial LVB size: 158, mtime: 0x6aa6977c, atime: 0x0, ctime: 0x6aa6977c, blocks: 0x1 00010000:00010000:1.0:1789302653.398926:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=490s v=(0 0) 00010000:00010000:1.0:1789302653.398931:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302653.398943:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.398965:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.398982:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea000, rc 0) 00000100:00100000:1.0:1789302653.399060:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350221824:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 250us (443us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.399095:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:3.0:1789302653.403678:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.403781:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350222080 00000100:00100000:1.0:1789302653.403795:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350222080 00000100:00100000:1.0:1789302653.403873:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 697 00000100:00100000:1.0:1789302653.403887:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:21817:x1876219350222080:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302653.404038:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:21817:x1876219350222080:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 154us (373us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.404047:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 697 00000100:00100000:3.0:1789302653.407741:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302653.407846:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350222208 00000100:00100000:0.0:1789302653.407861:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350222208 00002000:00100000:0.0:1789302653.407932:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (0->4095) req@ffff960f3f047b80 x1876219350222208/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:229/0 lens 488/0 e 0 to 0 dl 1789302664 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302653.407945:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f047b80 with x1876219350222208 ext(0->4095) 00002000:00010000:0.0:1789302653.407952:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f047b80 00000100:00100000:0.0:1789302653.407990:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 59 00000100:00100000:0.0:1789302653.408000:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350222208:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302653.415652:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350222208:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 7633us (7907us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302653.415693:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 59 00002000:00100000:0.0:1789302653.415714:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0xf:0x0] (0->4095) req@ffff960f3f047b80 x1876219350222208/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:229/0 lens 488/4536 e 0 to 0 dl 1789302664 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302653.415727:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f047b80 with x1876219350222208 ext(0->4095) 00002000:00010000:0.0:1789302653.415737:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f047b80 00000100:00100000:2.0:1789302653.421707:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.421819:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350222336 00000100:00100000:1.0:1789302653.421832:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350222336 00000100:00100000:1.0:1789302653.421905:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:1.0:1789302653.421918:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:21817:x1876219350222336:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302653.421947:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302653.421976:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c81b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff3d expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.422022:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.422039:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.422053:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.422246:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c81b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 37 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.422282:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.422292:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302653.422296:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302653.422301:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:1.0:1789302653.423744:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21817:x1876219350222336:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1815us (2038us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.423756:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:0.0:1789302653.431416:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302653.435432:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350222464 00000100:00100000:2.0:1789302653.435452:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350222464 00000100:00100000:0.0:1789302653.438793:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 699 00000100:00100000:0.0:1789302653.438812:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21817:x1876219350222464:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302653.438856:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x12:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302653.438996:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21817:x1876219350222464:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 189us (7595us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302653.439004:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 699 00000100:00100000:3.0:1789302653.445397:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302653.445463:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350222592 00000100:00100000:0.0:1789302653.445476:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350222592 00000100:00100000:0.0:1789302653.445609:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 178 00000100:00100000:0.0:1789302653.445622:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350222592:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302653.445683:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 3/1,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445704:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302653.445710:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302653.445717:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445749:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445788:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445798:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445808:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445816:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302653.445822:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445832:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445851:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302653.445859:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000ff4882b5/0x64fe43095184c829 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302653.445971:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350222592:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 350us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302653.445980:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 178 00000100:00100000:2.0:1789302653.451956:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.452031:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350222848 00000100:00100000:3.0:1789302653.452043:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350222848 00000100:00100000:3.0:1789302653.452131:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 700 00000100:00100000:3.0:1789302653.452143:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21817:x1876219350222848:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302653.452248:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.452271:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.452281:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.452322:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.452338:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 36 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302653.452356:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f047800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371059840:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302653.452467:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1840 going to sleep for 11 seconds 00000100:00100000:2.0:1789302653.453306:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371059968:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302653.453415:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302653.453476:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371059968 00000100:00100000:0.0:1789302653.453484:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371059968 00000100:00100000:0.0:1789302653.453542:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 126 00000100:00100000:0.0:1789302653.453551:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371059968:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302653.453740:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 57868288 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302653.453843:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371059968:12345-0@lo:13:osp-pre-1-0.0 Request processed in 293us (426us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302653.453850:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 126 00000100:00100000:2.0:1789302653.453932:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371059968:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302653.459568:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.459756:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f047800 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371059840:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302653.459793:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302653.459898:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350222976 00000100:00100000:1.0:1789302653.459906:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350222976 00010000:00010000:1.0:1789302653.459949:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 37 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302653.460023:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 193 00000100:00100000:1.0:1789302653.460034:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2348:x1876219350222976:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302653.460051:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302653.460058:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 37 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.460080:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 37 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.460122:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c822 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff3d expref: 37 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.460747:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302653.460756:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.460767:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.460777:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.460808:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302653.460875:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350222976:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 841us (1321us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.460884:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 193 00010000:00010000:3.0:1789302653.461005:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461026:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461289:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461299:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461306:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461312:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302653.461327:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461336:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461345:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302653.461348:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302653.461352:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.461359:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184c830 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302653.461490:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21817:x1876219350222848:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 9349us (9544us total) trans 4294967749 rc 0/0 00000100:00100000:3.0:1789302653.461523:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 700 00000040:00080000:3.0:1789302653.461656:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302653.461662:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302653.462235:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:3.0:1789302653.462246:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302653.466966:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.467103:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350223232 00000100:00100000:1.0:1789302653.467116:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350223232 00000100:00100000:1.0:1789302653.467196:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 142 00000100:00100000:1.0:1789302653.467209:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21817:x1876219350223232:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302653.467549:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21817:x1876219350223232:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 342us (595us total) trans 4294967750 rc 0/0 00000100:00100000:1.0:1789302653.467557:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 142 00010000:00010000:2.0:1789302653.582080:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302653.815675:0:17132:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302653.815686:0:17132:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302653.815735:0:17132:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 17 count 2 00000040:00080000:2.0:1789302653.815741:0:17132:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302653.876734:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.883917:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350223488 00000100:00100000:1.0:1789302653.883938:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350223488 00000100:00100000:1.0:1789302653.884024:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 701 00000100:00100000:1.0:1789302653.884048:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:21818:x1876219350223488:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:1.0:1789302653.884087:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302653.884116:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c837 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff44 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.884254:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c83e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.884273:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c83e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.884287:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c83e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302653.884303:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x12:0x0]: returning size 158 00010000:00010000:1.0:1789302653.884535:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c837 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff44 expref: 79 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.884555:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c83e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff44 expref: 78 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.884566:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302653.884570:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302653.884574:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb400, rc 0) 00000100:00100000:1.0:1789302653.884661:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21818:x1876219350223488:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 615us (7943us total) trans 4294967751 rc 0/0 00000100:00100000:1.0:1789302653.884668:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 701 00000100:00100000:1.0:1789302653.890594:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.890644:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.890738:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350223744 00000100:00100000:3.0:1789302653.890752:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350223744 00000100:00100000:3.0:1789302653.890837:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 29 00000100:00100000:3.0:1789302653.890850:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:2348:x1876219350223744:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302653.890871:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302653.890906:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c83e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff44 expref: 78 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.890928:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302653.890935:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302653.891005:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:2348:x1876219350223744:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 158us (428us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302653.891013:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 29 00000100:00100000:3.0:1789302653.891097:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350223616 00000100:00100000:3.0:1789302653.891102:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350223616 00000100:00100000:3.0:1789302653.891134:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 179 00000100:00100000:3.0:1789302653.891142:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350223616:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:3.0:1789302653.891159:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302653.891179:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 2/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff4b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.891205:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302653.891218:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302653.891227:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.891248:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.891262:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d400, rc 0) 00000100:00100000:3.0:1789302653.891286:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2345:x1876219350223616:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 145us (641us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302653.891292:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 179 00000100:00100000:1.0:1789302653.896182:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302653.896342:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350223872 00000100:00100000:3.0:1789302653.896356:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350223872 00002000:00100000:3.0:1789302653.896423:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (0->4095) req@ffff960f1db47800 x1876219350223872/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:229/0 lens 488/0 e 0 to 0 dl 1789302664 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302653.896438:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1db47800 with x1876219350223872 ext(0->4095) 00002000:00010000:3.0:1789302653.896444:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f1db47800 00000100:00100000:3.0:1789302653.896482:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 60 00000100:00100000:3.0:1789302653.896493:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350223872:12345-192.168.203.11@tcp:3:md5sum.0 00000100:00100000:3.0:1789302653.896790:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350223872:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 298us (617us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302653.896798:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 60 00002000:00100000:3.0:1789302653.896806:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0xf:0x0] (0->4095) req@ffff960f1db47800 x1876219350223872/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:229/0 lens 488/4536 e 0 to 0 dl 1789302664 ref 1 fl Complete:/600/0 rc 4096/4096 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302653.896814:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1db47800 with x1876219350223872 ext(0->4095) 00002000:00010000:3.0:1789302653.896832:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f1db47800 00000100:00100000:3.0:1789302653.900419:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.900553:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350224128 00000100:00100000:1.0:1789302653.900565:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350224128 00000100:00100000:1.0:1789302653.900632:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 143 00000100:00100000:1.0:1789302653.900644:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f07730850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:21818:x1876219350224128:12345-192.168.203.11@tcp:35:md5sum.0 00000100:00100000:1.0:1789302653.900958:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f07730850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21818:x1876219350224128:12345-192.168.203.11@tcp:35:md5sum.0 Request processed in 314us (551us total) trans 4294967752 rc 0/0 00000100:00100000:1.0:1789302653.900967:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 143 00000100:00100000:3.0:1789302653.911043:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.911155:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350224384 00000100:00100000:1.0:1789302653.911168:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350224384 00000100:00100000:1.0:1789302653.911239:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 702 00000100:00100000:1.0:1789302653.911251:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21819:x1876219350224384:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:1.0:1789302653.911326:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1789302653.911362:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184c7d5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff13 expref: 78 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911384:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302653.911396:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c7ea lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff21 expref: 77 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911407:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:1.0:1789302653.911410:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302653.911442:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911458:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911485:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911537:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911547:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911554:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911687:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911700:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911709:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911748:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.911765:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 36 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302653.911784:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371060096:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302653.911987:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a46c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302653.915284:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302653.915330:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350224512 00000100:00100000:1.0:1789302653.915343:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350224512 00010000:00010000:1.0:1789302653.915394:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 37 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302653.915435:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 194 00000100:00100000:1.0:1789302653.915447:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2347:x1876219350224512:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302653.915465:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302653.915475:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 37 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.915490:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 37 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.915542:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302653.915549:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.915561:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.915570:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.915597:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302653.915660:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2347:x1876219350224512:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 214us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302653.915668:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 194 00010000:00010000:3.0:1789302653.915926:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c7a4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x12:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fefe expref: 36 pid: 6919 timeout: 590 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302653.915946:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371060096:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302653.915971:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.915981:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.915987:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302653.917046:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].7, 64 off8640 00000004:00080000:3.0:1789302653.917062:0:6919:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 490927559286ns, next at 491927556200ns, rc = 0 00010000:00010000:3.0:1789302653.917171:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917188:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917201:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917210:0:6919:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917216:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917232:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917261:0:6919:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302653.917288:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917295:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917301:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917307:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302653.917312:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917320:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917332:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302653.917335:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302653.917340:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917345:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184c861 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x12:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917365:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917372:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917384:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917389:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302653.917392:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917399:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917407:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302653.917410:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302653.917413:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.917420:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184c853 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302653.917431:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000088ecaa4e reply state 0000000082014dd2 transno 4294967753 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302653.917441:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302653.917585:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21819:x1876219350224384:12345-192.168.203.11@tcp:36:rm.0 Request processed in 6333us (6555us total) trans 4294967753 rc 0/0 00000100:00100000:3.0:1789302653.917595:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 702 00010000:00010000:1.0:1789302653.917780:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302653.917789:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.917796:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302653.917800:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.917806:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.917817:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302653.917819:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789302653.917822:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.917828:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c868 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x12:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302653.917848:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302653.917850:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302653.917852:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302653.919934:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.919951:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.919974:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.919981:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302653.919987:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.919997:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.920017:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f10d24900) 00010000:00010000:3.0:1789302653.920020:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302653.920026:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302653.920033:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184c85a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302653.964776:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371060224:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302653.964879:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302653.965065:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371060224 00000100:00100000:0.0:1789302653.965085:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371060224 00000100:00100000:0.0:1789302653.965150:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:0.0:1789302653.965162:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371060224:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302653.965192:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 1703936 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302653.965252:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371060224:12345-0@lo:13:osp-pre-0-0.0 Request processed in 91us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302653.965260:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:3.0:1789302653.965365:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371060224:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302654.063599:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.064301:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350224640 00000100:00100000:1.0:1789302654.064316:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350224640 00000100:00100000:1.0:1789302654.064420:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 703 00000100:00100000:1.0:1789302654.064433:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:21835:x1876219350224640:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:1.0:1789302654.064473:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302654.064534:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c86f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff52 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302654.064807:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302654.064897:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c876 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.064928:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c876 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.064946:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c876 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302654.064991:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2b:0x0]: returning size 16384 00000004:00200000:1.0:1789302654.064996:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302654.065015:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c876 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.065045:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c86f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff52 expref: 79 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.065077:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c876 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff52 expref: 78 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.065088:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:1.0:1789302654.065092:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302654.065096:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:1.0:1789302654.065178:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21835:x1876219350224640:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 747us (1594us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.065186:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 703 00000100:00100000:1.0:1789302654.145710:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.145783:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350224768 00000100:00100000:3.0:1789302654.145798:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350224768 00000100:00100000:3.0:1789302654.145959:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 704 00000100:00100000:3.0:1789302654.145972:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:21841:x1876219350224768:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302654.146006:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302654.146064:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c87d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff59 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146170:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146188:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146201:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146218:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146225:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146231:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146602:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c892 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146612:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c892 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146620:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c892 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302654.146653:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00000004:00200000:3.0:1789302654.146658:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302654.146680:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c892 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146698:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146706:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146713:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146719:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302654.146725:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146734:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146748:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:3.0:1789302654.146752:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302654.146758:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146764:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c884 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146779:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146785:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146791:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146803:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2de00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302654.146806:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146812:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146820:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff960f0cdf2100) 00010000:00010000:3.0:1789302654.146823:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302654.146825:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146831:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c88b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146870:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c87d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff59 expref: 38 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146887:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2da00/0x64fe43095184c892 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff59 expref: 37 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.146895:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302654.146897:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302654.146900:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2da00, rc 0) 00000100:00100000:3.0:1789302654.146987:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21841:x1876219350224768:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1012us (1286us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302654.146996:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 704 00000100:00080000:0.0:1789302654.147637:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302654.151336:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.151573:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350224896 00000100:00100000:3.0:1789302654.151605:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350224896 00000100:00100000:3.0:1789302654.151736:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 705 00000100:00100000:3.0:1789302654.151749:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21841:x1876219350224896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302654.151777:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302654.151808:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c899 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff60 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.152032:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c899 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff60 expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.152048:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x12:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302654.152052:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302654.152058:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184c899 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x12:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff60 expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.152092:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:3.0:1789302654.152186:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21841:x1876219350224896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 439us (863us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302654.152193:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 705 00000100:00100000:2.0:1789302654.156831:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.157134:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350225152 00000100:00100000:3.0:1789302654.157156:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350225152 00000100:00100000:3.0:1789302654.157233:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 706 00000100:00100000:3.0:1789302654.157246:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:21841:x1876219350225152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302654.157285:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302654.157289:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789302654.157330:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184c876 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff52 expref: 78 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.157712:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302654.157730:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302654.157752:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184c8a0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff67 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.158019:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.158037:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.158049:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.158075:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.158083:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.158089:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.161246:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.161822:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162013:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302654.162052:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 0 00010000:00010000:3.0:1789302654.162103:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162112:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162120:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162127:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fae00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302654.162132:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162197:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162214:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:3.0:1789302654.162218:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302654.162225:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162232:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fae00/0x64fe43095184c8a7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302654.162247:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000044ed14dd reply state 000000006a4b42d4 transno 4294967754 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302654.162357:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302654.162563:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa800/0x64fe43095184c8a0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 80 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162599:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 79 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.162606:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302654.162609:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302654.162612:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f121dca00, rc 0) 00000100:00100000:3.0:1789302654.162735:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:21841:x1876219350225152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 5490us (5919us total) trans 4294967754 rc 0/0 00000100:00100000:3.0:1789302654.162746:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 706 00010000:00010000:1.0:1789302654.166693:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.166717:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.167221:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.167228:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6fa200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302654.167234:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.167243:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.167265:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f0cdf2f00) 00010000:00010000:1.0:1789302654.167269:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302654.167275:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.167283:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fa200/0x64fe43095184c8ae lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.167457:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.167670:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350225408 00000100:00100000:3.0:1789302654.167682:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350225408 00000100:00100000:3.0:1789302654.167729:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 707 00000100:00100000:3.0:1789302654.167737:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:21841:x1876219350225408:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302654.167761:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302654.167779:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184c8c3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff6e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.167890:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.167905:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 80 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.167916:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.167956:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 80 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.167983:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 80 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302654.168000:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371060352:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302654.168169:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1a80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302654.171946:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.172189:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044e00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371060352:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302654.172228:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.172472:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350225536 00000100:00100000:1.0:1789302654.172480:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350225536 00010000:00010000:1.0:1789302654.172566:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 81 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.172607:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 195 00000100:00100000:1.0:1789302654.172620:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:2347:x1876219350225536:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302654.172639:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302654.172647:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 81 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.172685:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 81 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.172713:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe43095184c8b5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff67 expref: 81 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.172731:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302654.172736:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.172748:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.172756:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.172788:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302654.172872:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.172872:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbe680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:2347:x1876219350225536:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 255us (940us total) trans 0 rc 0/0 00010000:00010000:3.0:1789302654.172880:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.172887:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 195 00000004:00200000:3.0:1789302654.172904:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 0 00010000:00010000:2.0:1789302654.175167:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.175199:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.175209:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.175216:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302654.175222:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.175231:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.175249:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:2.0:1789302654.175252:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302654.175268:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.175275:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184c8ca lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.176198:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184c8c3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff6e expref: 80 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.176214:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:2.0:1789302654.176217:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302654.176222:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe43095184c8c3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff6e expref: 80 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.176237:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f121ddc00, rc 0) 00000100:00100000:2.0:1789302654.177529:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:21841:x1876219350225408:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 9751us (10037us total) trans 4294967755 rc 301/301 00000100:00100000:2.0:1789302654.177544:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 707 00000100:00100000:2.0:1789302654.180967:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.181060:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350225792 00000100:00100000:1.0:1789302654.181086:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350225792 00000100:00100000:1.0:1789302654.181170:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 144 00000100:00100000:1.0:1789302654.181182:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:21841:x1876219350225792:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302654.182697:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:21841:x1876219350225792:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1507us (1735us total) trans 4294967756 rc 0/0 00000100:00100000:1.0:1789302654.182710:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 144 00000100:00100000:0.0:1789302654.185998:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.186105:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350225920 00000100:00100000:1.0:1789302654.186118:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350225920 00000100:00100000:1.0:1789302654.186228:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:1.0:1789302654.186241:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:21841:x1876219350225920:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302654.186333:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302654.186365:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c8d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff75 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.186656:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c8d8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff75 expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.186673:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c8d8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff75 expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.186692:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302654.186695:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302654.186699:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:1.0:1789302654.186767:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:21841:x1876219350225920:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 530us (783us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.186774:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:2.0:1789302654.189180:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.189315:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350226176 00000100:00100000:0.0:1789302654.189328:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350226176 00000100:00100000:2.0:1789302654.189549:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 709 00000100:00100000:2.0:1789302654.189563:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:21841:x1876219350226176:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1789302654.189651:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.189672:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.189687:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.189976:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.189999:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.190007:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.190014:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302654.190027:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.190036:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.190081:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302654.190084:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302654.190090:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.190096:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c8df lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.190208:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:21841:x1876219350226176:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 647us (1040us total) trans 4294967757 rc 0/0 00000100:00100000:2.0:1789302654.190216:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 709 00000100:00100000:2.0:1789302654.192382:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.193218:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350226432 00000100:00100000:3.0:1789302654.193232:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350226432 00000100:00100000:2.0:1789302654.195631:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 710 00000100:00100000:2.0:1789302654.195660:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:21841:x1876219350226432:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302654.196007:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.196835:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.196855:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302654.196974:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967758 00000004:00010000:2.0:1789302654.197056:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000b9c10e3 reply state 000000007ada76da transno 4294967759 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.197076:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302654.197401:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:21841:x1876219350226432:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1751us (5026us total) trans 4294967759 rc 0/0 00000100:00100000:2.0:1789302654.197418:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 710 00000020:00080000:2.0:1789302654.197424:0:5843:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1789302654.201774:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302654.201828:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.201845:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.201854:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.201861:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302654.201867:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.201899:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.201922:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302654.201925:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789302654.201932:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.201940:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184c8e6 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.202485:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350226688 00000100:00100000:2.0:1789302654.202497:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350226688 00000100:00100000:2.0:1789302654.203482:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 145 00000100:00100000:2.0:1789302654.203529:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:21841:x1876219350226688:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302654.203867:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:21841:x1876219350226688:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 373us (2109us total) trans 4294967760 rc 0/0 00000100:00100000:2.0:1789302654.203875:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 145 00000100:00100000:2.0:1789302654.233485:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.234101:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350226944 00000100:00100000:0.0:1789302654.234188:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350226944 00000100:00100000:0.0:1789302654.234432:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 711 00000100:00100000:0.0:1789302654.234445:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:21843:x1876219350226944:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:0.0:1789302654.234477:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302654.234534:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c8ed lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff7c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235476:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235499:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235557:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235585:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235601:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235608:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235772:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c902 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235782:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c902 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.235789:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c902 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302654.235815:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 158 00010000:00010000:0.0:1789302654.236049:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236057:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236076:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236083:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.236089:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236097:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236111:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:0.0:1789302654.236115:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.236120:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236126:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c8f4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236316:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236324:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236343:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236349:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.236353:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236360:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236368:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xc57698b5 (ffff960f09ae1100) 00010000:00010000:0.0:1789302654.236371:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.236374:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236380:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c8fb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2b:0x0].0xc57698b5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236458:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c8ed lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff7c expref: 88 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236475:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c902 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff7c expref: 87 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.236482:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:0.0:1789302654.236485:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302654.236488:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2800, rc 0) 00000100:00100000:0.0:1789302654.236633:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:21843:x1876219350226944:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 2190us (3162us total) trans 4294967761 rc 0/0 00000100:00100000:0.0:1789302654.236653:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 711 00000100:00100000:1.0:1789302654.240131:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.240349:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350227200 00000100:00100000:2.0:1789302654.240363:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350227200 00000100:00100000:2.0:1789302654.240443:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 30 00000100:00100000:2.0:1789302654.240454:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:21843:x1876219350227200:12345-192.168.203.11@tcp:103:md5sum.0 00010000:00010000:2.0:1789302654.240474:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1789302654.240543:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184c902 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff7c expref: 87 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.240571:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184c8d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff75 expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.240584:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302654.240598:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302654.240741:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:21843:x1876219350227200:12345-192.168.203.11@tcp:103:md5sum.0 Request processed in 287us (629us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.240770:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 30 00000100:00100000:2.0:1789302654.243765:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.244087:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350227328 00000100:00100000:1.0:1789302654.244100:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350227328 00000100:00100000:1.0:1789302654.244186:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 712 00000100:00100000:1.0:1789302654.244198:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:21843:x1876219350227328:12345-192.168.203.11@tcp:58:md5sum.0 00010000:00010000:1.0:1789302654.244335:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.244375:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.244389:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302654.244640:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].17, 136 off10504 00000100:00100000:1.0:1789302654.245445:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:21843:x1876219350227328:12345-192.168.203.11@tcp:58:md5sum.0 Request processed in 1241us (1687us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.245457:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 712 00000100:00100000:3.0:1789302654.248465:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.248575:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350227456 00000100:00100000:2.0:1789302654.248587:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350227456 00000100:00100000:2.0:1789302654.248708:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 713 00000100:00100000:2.0:1789302654.248830:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:21843:x1876219350227456:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:2.0:1789302654.248858:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302654.248886:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff83 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.248958:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.248971:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff83 expref: 86 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.248993:0:5844:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.249008:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff83 expref: 86 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.249035:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24900) 00010000:00010000:2.0:1789302654.249039:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302654.249043:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff83 expref: 86 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.249051:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf200, rc 0) 00000100:00100000:2.0:1789302654.249288:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:21843:x1876219350227456:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 569us (838us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.249296:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 713 00000040:00080000:3.0:1789302654.477790:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302654.478286:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302654.478373:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 18 count 3 00000040:00080000:3.0:1789302654.478382:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302654.478623:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371060480:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302654.481339:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302654.484627:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371060480:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302654.485948:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.490589:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350227584 00000100:00100000:2.0:1789302654.490611:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350227584 00000100:00100000:2.0:1789302654.490704:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 714 00000100:00100000:2.0:1789302654.490719:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21844:x1876219350227584:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302654.490766:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302654.491378:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21844:x1876219350227584:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 659us (5433us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.491388:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 714 00000100:00100000:3.0:1789302654.497960:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.502097:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350227712 00000100:00100000:0.0:1789302654.502117:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350227712 00000100:00100000:0.0:1789302654.502352:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 715 00000100:00100000:0.0:1789302654.502367:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21844:x1876219350227712:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1789302654.502476:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.502499:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.502532:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.502992:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503018:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503032:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503074:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503084:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503109:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302654.503166:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 158 00000004:00200000:0.0:1789302654.503171:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302654.503185:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503197:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503203:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503226:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503232:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.503237:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503245:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503257:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302654.503260:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.503265:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503272:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184c91e lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503298:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503305:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503311:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503317:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.503321:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503327:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503334:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x7d5aa0c7 (ffff960f09ae1100) 00010000:00010000:0.0:1789302654.503336:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.503339:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503345:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c925 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x7d5aa0c7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503364:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503371:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503377:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503388:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.503391:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503397:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503974:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24900) 00010000:00010000:0.0:1789302654.503978:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.503983:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.503990:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c92c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302654.504091:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21844:x1876219350227712:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 1724us (6685us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302654.504102:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 715 00000100:00100000:1.0:1789302654.509764:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.510075:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350227968 00000100:00100000:3.0:1789302654.510091:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350227968 00000100:00100000:3.0:1789302654.510180:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 716 00000100:00100000:3.0:1789302654.510192:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21844:x1876219350227968:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302654.510229:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302654.510257:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c933 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff8a expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510405:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510423:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510436:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302654.510449:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2c:0x0]: returning size 158 00010000:00010000:3.0:1789302654.510602:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510610:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510617:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510623:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302654.510628:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510636:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510650:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24900) 00010000:00010000:3.0:1789302654.510653:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302654.510659:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510676:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c93a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510737:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c933 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff8a expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510748:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302654.510751:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302654.510755:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c933 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff8a expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.510766:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2de00, rc 0) 00000100:00100000:3.0:1789302654.510852:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:21844:x1876219350227968:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 663us (1096us total) trans 4294967762 rc 301/301 00000100:00100000:3.0:1789302654.510860:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 716 00000100:00100000:3.0:1789302654.517450:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.517742:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350228096 00000100:00100000:2.0:1789302654.517755:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350228096 00000100:00100000:2.0:1789302654.517960:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 717 00000100:00100000:2.0:1789302654.517977:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:21844:x1876219350228096:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302654.518030:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302654.518195:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:21844:x1876219350228096:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 219us (757us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302654.521730:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.522447:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350228352 00000100:00100000:1.0:1789302654.522463:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350228352 00000100:00100000:2.0:1789302654.522820:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 717 00000100:00100000:1.0:1789302654.523751:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 718 00000100:00100000:1.0:1789302654.523770:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:21844:x1876219350228352:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302654.523798:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302654.523827:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184c948 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff91 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302654.526022:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0xf:0x0], index 1: 0 00010000:00010000:1.0:1789302654.529972:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c94f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.530002:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c94f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.530018:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c94f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.530404:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184c948 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff91 expref: 41 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.530430:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c94f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff91 expref: 40 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.530441:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f10d24100) 00010000:00010000:1.0:1789302654.530444:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302654.530597:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb000, rc 0) 00000100:00100000:1.0:1789302654.530808:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:21844:x1876219350228352:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 7044us (9103us total) trans 4294967763 rc 0/0 00000100:00100000:1.0:1789302654.530819:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 718 00000100:00100000:1.0:1789302654.537042:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.537163:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350228608 00000100:00100000:0.0:1789302654.537177:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350228608 00000100:00100000:0.0:1789302654.537267:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 719 00000100:00100000:0.0:1789302654.537280:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:21844:x1876219350228608:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302654.537361:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302654.537394:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c94f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff91 expref: 40 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.537419:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f10d24100) 00010000:00010000:0.0:1789302654.537433:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302654.537492:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.537556:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.537570:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538354:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538379:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538592:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538666:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538674:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538682:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538689:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.538693:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538701:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538736:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0x0 (ffff960f10d24c00) 00010000:00010000:0.0:1789302654.538739:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.538745:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538752:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184c95d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538767:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538773:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538778:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538784:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.538787:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538792:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538799:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2b:0x0].0xdfa00d87 (ffff960f09ae1100) 00010000:00010000:0.0:1789302654.538801:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.538804:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.538809:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c964 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2b:0x0].0xdfa00d87 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302654.538942:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:21844:x1876219350228608:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1660us (1910us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302654.538951:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 719 00000100:00100000:2.0:1789302654.548109:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.548192:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350228736 00000100:00100000:3.0:1789302654.548202:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350228736 00000100:00100000:3.0:1789302654.548265:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 720 00000100:00100000:3.0:1789302654.548275:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:21844:x1876219350228736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302654.548295:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302654.548319:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c96b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff98 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.548356:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.548383:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.548394:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.548485:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c96b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 41 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.548897:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 40 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.548907:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302654.548910:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302654.548913:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c400, rc 0) 00000100:00100000:2.0:1789302654.549045:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:21844:x1876219350228736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 770us (944us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.549053:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 720 00000100:00100000:3.0:1789302654.554769:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.555033:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350228992 00000100:00100000:0.0:1789302654.555044:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350228992 00000100:00100000:0.0:1789302654.555112:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 721 00000100:00100000:0.0:1789302654.555122:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:21844:x1876219350228992:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302654.555206:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.555224:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.555235:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789302654.555965:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].6, 88 off8624 00000004:00080000:0.0:1789302654.555979:0:5845:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 491566475670ns, next at 492566472563ns, rc = 0 00000004:00010000:0.0:1789302654.556083:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000057bc0d51 reply state 00000000d398a57c transno 4294967764 ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302654.556093:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302654.556198:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:21844:x1876219350228992:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1075us (1441us total) trans 4294967764 rc 0/0 00000100:00100000:0.0:1789302654.556206:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 721 00010000:00010000:0.0:1789302654.559374:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.559418:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.559426:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.559431:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.559436:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.559444:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.559459:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302654.559462:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789302654.559467:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.559472:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184c979 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302654.561152:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.561242:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350229120 00000100:00100000:1.0:1789302654.561257:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350229120 00000100:00100000:1.0:1789302654.561328:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 722 00000100:00100000:1.0:1789302654.561339:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:21844:x1876219350229120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302654.561363:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302654.561396:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ff9f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.561461:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 42 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.561476:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 42 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.561564:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302654.561575:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302654.561579:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:1.0:1789302654.561644:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:21844:x1876219350229120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 308us (512us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.561650:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 722 00000100:00100000:1.0:1789302654.567832:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.571214:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350229248 00000100:00100000:2.0:1789302654.571229:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350229248 00000100:00100000:2.0:1789302654.571298:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 180 00000100:00100000:2.0:1789302654.571322:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350229248:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00000100:00100000:2.0:1789302654.571930:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350229248:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 608us (4106us total) trans 4294967329 rc 0/0 00000100:00100000:2.0:1789302654.571939:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 180 00000100:00100000:3.0:1789302654.575410:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.575708:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350229504 00000100:00100000:1.0:1789302654.575731:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350229504 00000100:00100000:1.0:1789302654.575803:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 146 00000100:00100000:1.0:1789302654.575816:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:21844:x1876219350229504:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302654.576320:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:21844:x1876219350229504:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 504us (921us total) trans 4294967765 rc 0/0 00000100:00100000:1.0:1789302654.576329:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 146 00000100:00100000:3.0:1789302654.582472:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.582618:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350229760 00000100:00100000:0.0:1789302654.582630:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350229760 00000100:00100000:0.0:1789302654.582688:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 723 00000100:00100000:0.0:1789302654.582699:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:21844:x1876219350229760:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302654.583010:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:21844:x1876219350229760:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 302us (536us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.583026:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 723 00000100:00100000:2.0:1789302654.594382:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.594793:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350229888 00000100:00100000:1.0:1789302654.594803:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350229888 00000100:00100000:1.0:1789302654.594938:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 724 00000100:00100000:1.0:1789302654.594950:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:21844:x1876219350229888:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302654.594971:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302654.594991:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c987 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffa6 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.595096:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c98e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.595112:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c98e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.595123:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c98e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302654.595483:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302654.595531:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c98e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.595561:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184c987 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffa6 expref: 45 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.595833:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c98e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffa6 expref: 44 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.595936:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302654.595940:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1789302654.595944:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eae00, rc 0) 00000100:00100000:1.0:1789302654.596057:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:21844:x1876219350229888:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1108us (1683us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.596074:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 724 00000100:00100000:3.0:1789302654.602611:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.602729:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350230016 00000100:00100000:0.0:1789302654.602738:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350230016 00000100:00100000:0.0:1789302654.602792:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 181 00000100:00100000:0.0:1789302654.602801:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350230016:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302654.602833:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302654.602852:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffad expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302654.602876:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6aa6977e, blocks: 0x1 00010000:00010000:0.0:1789302654.602893:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=491s v=(0 0) 00010000:00010000:0.0:1789302654.602896:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302654.602907:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.602924:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.602935:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3c00, rc 0) 00000100:00100000:0.0:1789302654.602992:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350230016:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 192us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302654.602998:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 181 00010000:00010000:2.0:1789302654.604756:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302654.615082:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.615463:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350230272 00000100:00100000:1.0:1789302654.615473:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350230272 00000100:00100000:1.0:1789302654.615579:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 725 00000100:00100000:1.0:1789302654.615604:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:21844:x1876219350230272:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302654.615712:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:21844:x1876219350230272:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 111us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.615728:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 725 00000100:00100000:2.0:1789302654.625613:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.625740:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350230400 00000100:00100000:3.0:1789302654.625750:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350230400 00000100:00100000:3.0:1789302654.625808:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:3.0:1789302654.625818:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:21844:x1876219350230400:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302654.625836:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302654.625853:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 2/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffb4 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.625902:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.625911:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302654.625915:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302654.625921:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 4/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 12 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.625952:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.625965:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 12 pid: 6696 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302654.625979:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371060608:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302654.626079:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0defe900 going to sleep for 11 seconds 00000100:00100000:3.0:1789302654.632936:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.633078:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350230528 00000100:00100000:0.0:1789302654.633088:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350230528 00010000:00010000:0.0:1789302654.633122:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 13 pid: 6696 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302654.633173:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 196 00000100:00100000:0.0:1789302654.633183:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350230528:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302654.633209:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302654.633247:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 13 pid: 6696 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.633272:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 13 pid: 6696 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.633297:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302654.633301:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302654.633309:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 13 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.633319:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 13 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.633357:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 13 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302654.633370:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef32a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371060736:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302654.633446:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf600 going to sleep for 11 seconds 00010000:00010000:2.0:1789302654.637298:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bb0ec6bc/0x64fe43095184c995 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffad expref: 13 pid: 6696 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.637323:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371060608:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302654.637353:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 12 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.637367:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:2.0:1789302654.637438:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:21844:x1876219350230400:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 11620us (11833us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.637446:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:1.0:1789302654.638044:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef32a00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371060736:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302654.638057:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302654.638100:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350230528:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4919us (5174us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.638108:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 196 00000100:00100000:3.0:1789302654.641093:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.648873:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350230656 00000100:00100000:0.0:1789302654.648886:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350230656 00002000:00100000:0.0:1789302654.648941:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xf:0x0] (0->157) req@ffff960f3ad44a80 x1876219350230656/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:230/0 lens 648/0 e 0 to 0 dl 1789302665 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302654.648953:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad44a80 with x1876219350230656 ext(0->157) 00002000:00010000:0.0:1789302654.648958:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ad44a80 00000100:00100000:0.0:1789302654.648990:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 61 00000100:00100000:0.0:1789302654.649012:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350230656:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1789302654.649459:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350230656:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 447us (8374us total) trans 4294967330 rc 0/0 00000100:00100000:0.0:1789302654.649467:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 61 00002000:00100000:0.0:1789302654.649474:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0xf:0x0] (0->157) req@ffff960f3ad44a80 x1876219350230656/t4294967330(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:230/0 lens 648/448 e 0 to 0 dl 1789302665 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302654.649483:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad44a80 with x1876219350230656 ext(0->157) 00002000:00010000:0.0:1789302654.649486:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ad44a80 00000100:00100000:2.0:1789302654.659077:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.670742:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350230784 00000100:00100000:1.0:1789302654.670757:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350230784 00000100:00100000:1.0:1789302654.670841:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 726 00000100:00100000:1.0:1789302654.670853:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:21844:x1876219350230784:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1789302654.743909:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967764 is committed 00000004:00080000:2.0:1789302654.743921:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 7 committed 00000001:00080000:2.0:1789302654.743934:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967761 is committed 00000001:00080000:2.0:1789302654.743943:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967765 is committed 00000004:00080000:2.0:1789302654.743982:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 8 committed 00000100:00100000:2.0:1789302654.744343:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371060864:0@lo:2:osp-syn-1-0.0 00000100:00100000:2.0:1789302654.744419:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302654.744485:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371060992:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789302654.744531:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302654.744564:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371060864 00000100:00100000:2.0:1789302654.744570:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371060864 00000100:00100000:2.0:1789302654.744609:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371060992 00000100:00100000:2.0:1789302654.744612:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371060992 00000100:00100000:2.0:1789302654.744629:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:2.0:1789302654.744636:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371060864:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:2.0:1789302654.744780:0:6696:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:2.0:1789302654.744784:0:6696:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 0 -> 1789302654 00002000:00010000:2.0:1789302654.744786:0:6696:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:2.0:1789302654.744788:0:6696:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789302654 -> 1789302654 00002000:00010000:2.0:1789302654.744791:0:6696:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:2.0:1789302654.744829:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371060864:12345-0@lo:2:osp-syn-1-0.0 Request processed in 192us (409us total) trans 4294967331 rc 0/0 00000100:00100000:2.0:1789302654.744836:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:2.0:1789302654.744853:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:2.0:1789302654.744859:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371060992:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789302654.744872:0:6696:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:15 count 1 00010000:00010000:2.0:1789302654.744890:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.744910:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 10 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.744920:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 2/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.744934:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789302654.744938:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302654.744943:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 5/0,1 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.744968:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.744982:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 9 pid: 6697 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.744993:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371061120:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302654.745050:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 10 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.745060:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 10 pid: 8571 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.745082:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371061248:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302654.745101:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75f00 going to sleep for 11 seconds 00000100:00080000:2.0:1789302654.745113:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302654.745117:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000004:00080000:0.0:1789302654.748077:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ffa4700/2, rc 0, transno 4294967331 00000100:00100000:0.0:1789302654.748090:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371060864:0@lo:2:osp-syn-1-0.0 00000100:00100000:3.0:1789302654.751790:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21844:x1876219350230784:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 80856us (92997us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302654.751807:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 726 00000100:00100000:0.0:1789302654.757761:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302654.757794:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.757829:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350230912 00000100:00100000:0.0:1789302654.757835:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.757839:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350230912 00000100:00080000:0.0:1789302654.757874:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789302654.757885:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 11 pid: 8571 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.757927:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350231040 00000100:00100000:1.0:1789302654.757932:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350231040 00010000:00010000:1.0:1789302654.757947:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 10 pid: 6697 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.757967:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 197 00000100:00100000:1.0:1789302654.757980:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350230912:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302654.758010:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302654.758017:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 11 pid: 8571 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.758029:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 11 pid: 8571 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.758053:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302654.758109:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350230912:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 133us (357us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.758116:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 197 00000100:00100000:3.0:1789302654.758251:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 198 00000100:00100000:3.0:1789302654.758262:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350231040:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302654.758271:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302654.758282:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 5/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 10 pid: 6697 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.758294:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 4/0,0 mode: PR/PR res: [0x240000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 10 pid: 6697 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.758310:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302654.758314:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302654.758320:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.758329:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.758347:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302654.758368:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350231040:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 108us (573us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302654.758379:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 198 00000100:00100000:2.0:1789302654.758535:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350231168 00000100:00100000:2.0:1789302654.758540:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350231168 00000100:00100000:2.0:1789302654.758571:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 185 00000100:00100000:2.0:1789302654.758578:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350231168:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00010000:00010000:2.0:1789302654.760025:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f4a07275/0x64fe43095184c84c lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff4b expref: 9 pid: 6697 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.760043:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371061120:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302654.760055:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c17c244b/0x64fe43095184c814 lrc: 0/0,0 mode: --/PR res: [0x240000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff36 expref: 10 pid: 8571 timeout: 591 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.760075:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371061248:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302654.760097:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.760108:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.760116:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.760139:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.760147:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.760185:0:6696:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789302654.760813:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371060992:12345-0@lo:6:osp-syn-0-0.0 Request processed in 15952us (16280us total) trans 4294967329 rc 0/0 00000100:00100000:2.0:1789302654.760823:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000004:00080000:2.0:1789302654.760874:0:3300:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ffa4000/2, rc 0, transno 4294967329 00000100:00100000:2.0:1789302654.760881:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371060992:0@lo:6:osp-syn-0-0.0 00010000:00010000:1.0:1789302654.761060:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302654.761078:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.761086:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302654.761090:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.761099:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.761115:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 1/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.761121:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000030d463c9/0x64fe43095184c9a3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.761146:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302654.761148:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302654.761150:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789302654.809494:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967330 is committed 00000100:00100000:2.0:1789302654.809858:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350231168:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 51274us (52016us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.809870:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 185 00000001:00080000:0.0:1789302654.810038:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967331 is committed 00000100:00100000:0.0:1789302654.829268:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.829399:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350231424 00000100:00100000:1.0:1789302654.829408:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350231424 00000100:00100000:1.0:1789302654.829468:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 727 00000100:00100000:1.0:1789302654.829478:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21844:x1876219350231424:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302654.829567:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302654.829697:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eae00/0x64fe43095184c98e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffa6 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.829717:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302654.829722:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302654.829750:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.829763:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.829772:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302654.830177:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000040bce94 reply state 00000000e1f13b17 transno 4294967766 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.830187:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302654.830298:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:21844:x1876219350231424:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 821us (1038us total) trans 4294967766 rc 0/0 00000100:00100000:2.0:1789302654.830305:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 727 00010000:00010000:0.0:1789302654.832315:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.832329:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.832335:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.832341:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302654.832360:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.832367:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.832380:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302654.832383:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302654.832387:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 2/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.832393:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea000/0x64fe43095184c9aa lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302654.835445:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.835659:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350231552 00000100:00100000:2.0:1789302654.835669:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350231552 00000100:00100000:2.0:1789302654.835727:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 186 00000100:00100000:2.0:1789302654.835736:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350231552:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1789302654.835986:0:6695:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:2.0:1789302654.835989:0:6695:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb mtime from disk: 1789302654 -> 1789302652 00002000:00010000:2.0:1789302654.835991:0:6695:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb atime from disk: 0 -> 1789302652 00002000:00010000:2.0:1789302654.835993:0:6695:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb ctime from disk: 1789302654 -> 1789302653 00002000:00010000:2.0:1789302654.835995:0:6695:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0xf:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1789302654.836077:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350231552:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 342us (640us total) trans 4294967332 rc 0/0 00000100:00100000:2.0:1789302654.836085:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 186 00000100:00100000:3.0:1789302654.845377:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.845551:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350231808 00000100:00100000:2.0:1789302654.845563:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350231808 00000100:00100000:2.0:1789302654.845619:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 728 00000100:00100000:2.0:1789302654.845628:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:21844:x1876219350231808:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1789302654.845691:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.845707:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 34 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.845716:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.845758:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 34 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.845773:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 34 pid: 6919 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.845787:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371061376:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302654.845867:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75000 going to sleep for 11 seconds 00000100:00100000:2.0:1789302654.850672:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371061376:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302654.850702:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302654.852277:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.852485:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350231936 00000100:00100000:1.0:1789302654.852495:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350231936 00010000:00010000:1.0:1789302654.852572:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 35 pid: 6919 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.852604:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 199 00000100:00100000:1.0:1789302654.852630:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:2348:x1876219350231936:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302654.852644:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302654.852651:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 35 pid: 6919 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.852665:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 35 pid: 6919 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.852685:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184c972 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x13:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff98 expref: 35 pid: 6919 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.852700:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302654.852704:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.852713:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.852721:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.852745:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302654.852792:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2348:x1876219350231936:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 164us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.852799:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 199 00010000:00010000:2.0:1789302654.853885:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853898:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853927:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853934:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853941:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853947:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302654.853951:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853959:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853972:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:2.0:1789302654.853974:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302654.853979:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.853985:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfe00/0x64fe43095184c9b1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.854074:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:21844:x1876219350231808:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 8435us (8695us total) trans 0 rc -22/-22 00000100:00100000:2.0:1789302654.854095:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 728 00000100:00100000:3.0:1789302654.862527:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.863153:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350232064 00000100:00100000:2.0:1789302654.863164:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350232064 00000100:00100000:2.0:1789302654.863226:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 187 00000100:00100000:2.0:1789302654.863237:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350232064:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302654.863287:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/1,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.863309:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.863319:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302654.863322:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302654.863327:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 4/1,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.863362:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.863376:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 11 pid: 13439 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.863392:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371061504:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302654.863474:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75000 going to sleep for 11 seconds 00000100:00100000:0.0:1789302654.878944:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.880577:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350232192 00000100:00100000:3.0:1789302654.880589:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350232192 00010000:00010000:3.0:1789302654.880644:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 12 pid: 13439 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302654.880692:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 200 00000100:00100000:3.0:1789302654.880702:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350232192:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302654.880744:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302654.880768:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 5/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 12 pid: 13439 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.880782:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 12 pid: 13439 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.880805:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302654.880810:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302654.880815:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.880824:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.880844:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302654.880922:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350232192:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 222us (1987us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302654.880930:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 200 00010000:00010000:2.0:1789302654.882423:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f6c72544/0x64fe43095184c99c lrc: 0/0,0 mode: --/PW res: [0x280000400:0xf:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffb4 expref: 11 pid: 13439 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.882460:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371061504:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302654.882484:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882494:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882546:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882578:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882587:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882594:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882600:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfe00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302654.882604:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882613:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882635:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.882643:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9b8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.882714:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350232064:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 19478us (20210us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.882735:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 187 00000100:00100000:0.0:1789302654.890187:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.890271:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350232448 00000100:00100000:2.0:1789302654.890281:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350232448 00000100:00100000:2.0:1789302654.890347:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 729 00000100:00100000:2.0:1789302654.890356:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:21844:x1876219350232448:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302654.890449:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.890467:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.890479:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.890745:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.890758:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.890785:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.890818:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.890832:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 34 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.890848:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371061632:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302654.890935:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75000 going to sleep for 11 seconds 00000100:00100000:3.0:1789302654.900116:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302654.900225:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350232576 00000100:00100000:1.0:1789302654.900236:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350232576 00010000:00010000:1.0:1789302654.900279:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 35 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.900343:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 201 00000100:00100000:1.0:1789302654.900352:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:2347:x1876219350232576:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302654.900368:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302654.900375:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 35 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.900392:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 35 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.900414:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302654.900419:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.900427:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.900434:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.900451:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302654.900534:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:2347:x1876219350232576:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 182us (433us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302654.900542:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 201 00010000:00010000:2.0:1789302654.900695:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184c980 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff9f expref: 34 pid: 5843 timeout: 591 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.900713:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371061632:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302654.900732:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.900740:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.900745:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302654.900912:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967767 00000001:00080000:2.0:1789302654.901079:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967768 00000001:00080000:2.0:1789302654.901125:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967769 00000100:00100000:2.0:1789302654.901213:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371061760:0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789302654.902556:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302654.903181:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75f00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302654.905454:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371061760 00000100:00100000:0.0:1789302654.905468:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371061760 00000100:00100000:0.0:1789302654.905550:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 23 00000100:00100000:0.0:1789302654.905561:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371061760:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302654.905696:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371061760:12345-0@lo:1000:mdt00_001.0 Request processed in 137us (3141us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302654.905703:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 23 00000100:00100000:1.0:1789302654.905947:0:5844:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967331 gen 2 00000004:00080000:1.0:1789302654.905952:0:5844:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ffa4700, transno 4294967331 00000100:00100000:1.0:1789302654.905972:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371061760:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1789302654.907591:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967770 00000100:00100000:1.0:1789302654.907676:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f754000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371061888:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1789302654.907935:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302654.907997:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302654.912712:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371061888 00000100:00100000:2.0:1789302654.912805:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371061888 00000100:00100000:2.0:1789302654.912920:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 24 00000100:00100000:2.0:1789302654.912932:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371061888:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789302654.913242:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5844:x1876219371061888:12345-0@lo:1000:mdt00_001.0 Request processed in 309us (5304us total) trans 4294967333 rc 0/0 00000100:00100000:2.0:1789302654.913249:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 24 00000100:00100000:1.0:1789302654.913336:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f754000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371061888:0@lo:1000:mdt00_001.0 00010000:00010000:1.0:1789302654.913381:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913391:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913408:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913414:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302654.913418:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913426:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913440:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302654.913443:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302654.913447:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913453:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184c9c6 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x13:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913482:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913532:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913539:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913543:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302654.913546:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913551:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913560:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24900) 00010000:00010000:1.0:1789302654.913563:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302654.913569:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff83 expref: 73 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913579:0:5844:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff83 expref: 73 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913588:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff83 expref: 73 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.913625:0:5844:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf200/0x64fe43095184c917 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ff83 expref: 73 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.913639:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371062016:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302654.914004:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302654.918314:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.918346:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350232704 00000100:00100000:3.0:1789302654.918354:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350232704 00000100:00100000:3.0:1789302654.918409:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 147 00000100:00100000:3.0:1789302654.918418:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21843:x1876219350232704:12345-192.168.203.11@tcp:49:md5sum.0 00000100:00100000:3.0:1789302654.918548:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21843:x1876219350232704:12345-192.168.203.11@tcp:49:md5sum.0 Request processed in 131us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302654.918568:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 147 00000100:00100000:1.0:1789302654.918843:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371062016:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302654.918870:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918890:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184c910 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918921:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918933:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918944:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918953:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918959:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918965:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918971:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302654.918975:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.918982:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919004:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24900) 00010000:00010000:1.0:1789302654.919007:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302654.919011:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919017:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184c9cd lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919026:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919032:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919038:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919043:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302654.919046:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919052:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919058:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2c:0x0].0x0 (ffff960f10d24900) 00010000:00010000:1.0:1789302654.919071:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302654.919074:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302654.919080:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184c9bf lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302654.919289:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21844:x1876219350232448:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 28933us (29110us total) trans 4294967771 rc 0/0 00000100:00100000:1.0:1789302654.919299:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 729 00000020:00080000:1.0:1789302654.919303:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:3.0:1789302654.919622:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302654.919626:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302654.925937:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:3.0:1789302654.925956:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302654.926578:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.932630:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350232960 00000100:00100000:3.0:1789302654.932645:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350232960 00000100:00100000:3.0:1789302654.932768:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 148 00000100:00100000:3.0:1789302654.932778:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:21844:x1876219350232960:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302654.933205:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000004:00080000:3.0:1789302654.938733:0:5847:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x13:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1789302654.938858:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967772 00010000:00010000:3.0:1789302654.939085:0:5847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.939102:0:5847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.939113:0:5847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.939121:0:5847:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.939128:0:5847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.939238:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350233088 00000100:00100000:2.0:1789302654.939249:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350233088 00010000:00010000:3.0:1789302654.939259:0:5847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302654.939279:0:5847:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789302654.939309:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 188 00000100:00100000:2.0:1789302654.939318:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:21843:x1876219350233088:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:2.0:1789302654.939332:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302654.939345:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9db lrc: 2/0,0 mode: --/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffbb expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302654.939369:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0xf:0x0] initial LVB size: 158, mtime: 0x6aa6977c, atime: 0x6aa6977c, ctime: 0x6aa6977d, blocks: 0xb 00000100:00100000:3.0:1789302654.939374:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:21844:x1876219350232960:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 6598us (12801us total) trans 4294967773 rc 0/0 00000100:00100000:3.0:1789302654.939382:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 148 00000020:00080000:3.0:1789302654.939386:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1789302654.939389:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=491s v=(0 0) 00010000:00010000:2.0:1789302654.939466:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0xf:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302654.939468:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302654.939474:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.939476:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffbb expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.939482:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2de00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302654.939490:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000026fdb211/0x64fe43095184c9db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0xf:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffbb expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.939492:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.939521:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302654.939530:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfe00, rc 0) 00010000:00010000:0.0:1789302654.939532:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x13:0x0].0x0 (ffff960f0cdf2100) 00010000:00010000:0.0:1789302654.939535:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302654.939539:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302654.939545:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184c9d4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x13:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302654.939550:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:21843:x1876219350233088:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 234us (6347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302654.939556:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 188 00010000:00010000:0.0:1789302654.939561:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302654.939563:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302654.939564:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302654.951803:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302654.954341:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350233216 00000100:00100000:2.0:1789302654.954357:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350233216 00002000:00100000:2.0:1789302654.954837:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xf:0x0] (0->4095) req@ffff960f3ffbe680 x1876219350233216/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:230/0 lens 488/0 e 0 to 0 dl 1789302665 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302654.954848:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffbe680 with x1876219350233216 ext(0->4095) 00002000:00010000:2.0:1789302654.954854:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ffbe680 00000100:00100000:2.0:1789302654.954942:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 62 00000100:00100000:2.0:1789302654.954953:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350233216:12345-192.168.203.11@tcp:3:md5sum.0 00000100:00100000:2.0:1789302654.955219:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350233216:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 268us (3425us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1789302654.955227:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 62 00002000:00100000:2.0:1789302654.955234:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0xf:0x0] (0->4095) req@ffff960f3ffbe680 x1876219350233216/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:230/0 lens 488/4536 e 0 to 0 dl 1789302665 ref 1 fl Complete:/600/0 rc 4096/4096 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302654.955243:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffbe680 with x1876219350233216 ext(0->4095) 00002000:00010000:2.0:1789302654.955247:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ffbe680 00000100:00100000:1.0:1789302654.964637:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302654.964723:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350233472 00000100:00100000:3.0:1789302654.964734:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350233472 00000100:00100000:3.0:1789302654.964786:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 149 00000100:00100000:3.0:1789302654.964797:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ed850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:21843:x1876219350233472:12345-192.168.203.11@tcp:35:md5sum.0 00000100:00100000:3.0:1789302654.965136:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ed850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:21843:x1876219350233472:12345-192.168.203.11@tcp:35:md5sum.0 Request processed in 340us (508us total) trans 4294967774 rc 0/0 00000100:00100000:3.0:1789302654.965149:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 149 00010000:00010000:1.0:1789302655.628812:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789302656.654083:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967328 is committed 00000001:00080000:0.0:1789302656.654126:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967329 is committed 00010000:00010000:3.0:1789302656.661061:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302657.676748:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302658.062007:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371062144:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062122:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302658.062218:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371062144 00000100:00100000:2.0:1789302658.062246:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371062144 00000100:00100000:2.0:1789302658.062330:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:2.0:1789302658.062340:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371062144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062401:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371062144:12345-0@lo:400:kworker.0 Request processed in 63us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302658.062409:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:2.0:1789302658.062467:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371062144:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062538:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371062272:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062571:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302658.062606:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371062272 00000100:00100000:2.0:1789302658.062610:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371062272 00000100:00100000:2.0:1789302658.062642:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:2.0:1789302658.062649:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371062272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062685:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371062272:12345-0@lo:400:kworker.0 Request processed in 37us (114us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302658.062692:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:2.0:1789302658.062735:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371062272:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062763:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302658.062770:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371062400:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062792:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302658.062817:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371062400 00000100:00100000:2.0:1789302658.062821:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371062400 00000100:00100000:2.0:1789302658.062862:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1789302658.062868:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371062400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062898:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371062400:12345-0@lo:400:kworker.0 Request processed in 30us (105us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302658.062911:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1789302658.062933:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371062400:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062959:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371062528:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.062984:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302658.063008:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371062528 00000100:00100000:2.0:1789302658.063011:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371062528 00000100:00100000:2.0:1789302658.063054:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1789302658.063061:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371062528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.063091:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371062528:12345-0@lo:400:kworker.0 Request processed in 31us (107us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302658.063097:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:2.0:1789302658.063115:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967329 gen 2 00000004:00080000:2.0:1789302658.063120:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ffa4000, transno 4294967329 00000100:00100000:2.0:1789302658.063131:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371062528:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.063536:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371062656:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.063561:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302658.063587:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371062656 00000100:00100000:2.0:1789302658.063591:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371062656 00000100:00100000:2.0:1789302658.063612:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:2.0:1789302658.063617:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371062656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.063646:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371062656:12345-0@lo:400:kworker.0 Request processed in 29us (84us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302658.063653:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00100000:2.0:1789302658.063676:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371062656:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.063718:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371062784:0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.063743:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302658.063767:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371062784 00000100:00100000:2.0:1789302658.063770:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371062784 00000100:00100000:2.0:1789302658.063794:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:2.0:1789302658.063800:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371062784:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302658.063828:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371062784:12345-0@lo:400:kworker.0 Request processed in 28us (84us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302658.063834:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:2.0:1789302658.063856:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371062784:0@lo:400:kworker.0 00000100:00080000:2.0:1789302658.063918:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302658.063922:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302658.063925:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302658.320648:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302658.320692:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302658.320720:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302658.320744:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302658.320764:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302658.320783:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302658.320806:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302658.320844:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350233728 00000100:00100000:1.0:1789302658.320858:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350233728 00000100:00100000:1.0:1789302658.320925:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350234112 00000100:00100000:1.0:1789302658.320928:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350234112 00000100:00100000:1.0:1789302658.320953:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:1.0:1789302658.320966:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2345:x1876219350233728:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302658.321075:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2345:x1876219350233728:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 112us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302658.321083:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:1.0:1789302658.321104:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:1.0:1789302658.321127:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2345:x1876219350234112:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302658.321158:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2345:x1876219350234112:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302658.321165:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:1.0:1789302658.321183:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350233856 00000100:00100000:1.0:1789302658.321186:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350233856 00000100:00100000:1.0:1789302658.321212:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350233984 00000100:00100000:1.0:1789302658.321215:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350233984 00000100:00100000:1.0:1789302658.321228:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350234240 00000100:00100000:1.0:1789302658.321230:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350234240 00000100:00100000:1.0:1789302658.321241:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350234368 00000100:00100000:1.0:1789302658.321243:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350234368 00000100:00100000:1.0:1789302658.321258:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 269 00000100:00100000:1.0:1789302658.321264:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350233856:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302658.321290:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350233856:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (569us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302658.321296:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 269 00000100:00100000:1.0:1789302658.321307:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 270 00000100:00100000:1.0:1789302658.321313:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350233984:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302658.321335:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350233984:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (591us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302658.321341:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 270 00000100:00100000:1.0:1789302658.321352:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 271 00000100:00100000:1.0:1789302658.321358:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350234240:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302658.321378:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350234240:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (594us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302658.321391:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 271 00000100:00100000:1.0:1789302658.321404:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 272 00000100:00100000:1.0:1789302658.321410:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350234368:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302658.321436:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350234368:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (630us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302658.321442:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 272 00000100:00080000:1.0:1789302658.321474:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302658.321480:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302658.321482:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302658.322722:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350233600 00000100:00100000:0.0:1789302658.322736:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350233600 00000100:00100000:0.0:1789302658.322788:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 355 00000100:00100000:0.0:1789302658.322798:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350233600:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302658.322864:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350233600:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (2231us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302658.322872:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 355 00000100:00100000:3.0:1789302658.577718:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302658.578641:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371062912:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302658.578738:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302658.585427:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371062912 00000100:00100000:0.0:1789302658.585450:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371062912 00000100:00100000:0.0:1789302658.585561:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 128 00000100:00100000:0.0:1789302658.585573:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371062912:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302658.585645:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302658.585762:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371062912:12345-0@lo:13:osp-pre-1-0.0 Request processed in 191us (7024us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302658.585770:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 128 00000100:00100000:2.0:1789302658.586082:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371062912:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302658.702776:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302659.020952:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302659.020967:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302659.021022:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 18 count 3 00000040:00080000:0.0:1789302659.021040:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302659.086944:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302659.086975:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371063040:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302659.087096:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302659.087331:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063040 00000100:00100000:0.0:1789302659.087343:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063040 00000100:00100000:0.0:1789302659.087422:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:0.0:1789302659.087433:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371063040:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302659.087482:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302659.087573:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371063040:12345-0@lo:13:osp-pre-0-0.0 Request processed in 141us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302659.087581:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:3.0:1789302659.087839:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371063040:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302659.725647:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302660.204647:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967771 is committed 00000020:00080000:2.0:1789302660.204667:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f0cf61000 refcount 2 committed 0 result 0 batchid 10 00000020:00080000:2.0:1789302660.204673:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f12140100 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f0cf61e40 00000020:00080000:2.0:1789302660.204677:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f12140500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3b085000 00000001:00080000:2.0:1789302660.204710:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967774 is committed 00000001:00080000:2.0:1789302660.204717:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967773 is committed 00000020:00080000:2.0:1789302660.204731:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 10 committed batchid 9 00000020:00080000:2.0:1789302660.205039:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 9 00000001:00080000:3.0:1789302660.235159:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967333 is committed 00000001:00080000:3.0:1789302660.235183:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967332 is committed 00000100:00100000:1.0:1789302660.678710:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302660.679235:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350234496 00000100:00100000:2.0:1789302660.679248:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350234496 00000100:00080000:1.0:1789302660.679376:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302660.679912:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 730 00000100:00100000:2.0:1789302660.679929:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21800:x1876219350234496:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1789302660.680101:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21800:x1876219350234496:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 178us (1404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302660.680110:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 730 00000100:00100000:2.0:1789302660.692645:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302660.692741:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350234752 00000100:00100000:3.0:1789302660.692755:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350234752 00000100:00100000:3.0:1789302660.692863:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 150 00000100:00100000:3.0:1789302660.692876:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:21800:x1876219350234752:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302660.693289:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:21800:x1876219350234752:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 415us (661us total) trans 4294967775 rc 0/0 00000100:00100000:3.0:1789302660.694659:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 150 00010000:00010000:3.0:1789302660.750741:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302661.772857:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1789302662.688473:0:17338:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 11c: Import a file to a directory with a pool ========================================================== 08:31:00 (1789302660) 00010000:00010000:3.0:1789302662.802472:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302663.180829:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371063552:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.180898:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302663.180970:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789302663.180977:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371063168:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181015:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302663.181041:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371063680:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181066:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302663.181090:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371063424:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181113:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302663.181131:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789302663.181149:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371063296:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181168:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302663.181180:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371063808:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181195:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302663.181216:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063168 00000100:00100000:0.0:1789302663.181222:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063168 00000100:00100000:0.0:1789302663.181272:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1789302663.181285:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371063168:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181326:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371063168:12345-0@lo:400:kworker.0 Request processed in 46us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302663.181332:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:0.0:1789302663.181366:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063680 00000100:00100000:0.0:1789302663.181370:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063680 00000100:00100000:0.0:1789302663.181398:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063424 00000100:00100000:0.0:1789302663.181401:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063424 00000100:00100000:0.0:1789302663.181412:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063296 00000100:00100000:0.0:1789302663.181414:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063296 00000100:00100000:0.0:1789302663.181426:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1789302663.181432:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371063680:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181464:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371063680:12345-0@lo:400:kworker.0 Request processed in 32us (397us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302663.181469:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:0.0:1789302663.181482:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:0.0:1789302663.181488:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371063424:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181542:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371063424:12345-0@lo:400:kworker.0 Request processed in 54us (428us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302663.181547:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:0.0:1789302663.181568:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1789302663.181574:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371063296:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181604:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371063296:12345-0@lo:400:kworker.0 Request processed in 31us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302663.181610:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1789302663.181647:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371063296:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181658:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371063424:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181687:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371063168:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.181696:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371063680:0@lo:400:kworker.0 00000100:00100000:2.0:1789302663.181836:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063552 00000100:00100000:2.0:1789302663.181841:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063552 00000100:00100000:2.0:1789302663.181875:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063808 00000100:00100000:2.0:1789302663.181878:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063808 00000100:00100000:2.0:1789302663.181897:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:2.0:1789302663.181904:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371063552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302663.181938:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371063552:12345-0@lo:400:kworker.0 Request processed in 35us (1041us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302663.181944:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00100000:2.0:1789302663.181954:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:2.0:1789302663.181959:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371063808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302663.181980:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371063808:12345-0@lo:400:kworker.0 Request processed in 21us (785us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302663.181986:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00080000:1.0:1789302663.182075:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302663.182079:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302663.182082:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302663.183541:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967333 gen 2 00000004:00080000:0.0:1789302663.183552:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967333 committed transno 4294967333 00000100:00100000:0.0:1789302663.183587:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371063808:0@lo:400:kworker.0 00000100:00100000:0.0:1789302663.183619:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371063552:0@lo:400:kworker.0 00000100:00100000:3.0:1789302663.436357:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302663.436396:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302663.436466:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302663.436485:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302663.436548:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350234880 00000100:00100000:0.0:1789302663.436557:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350234880 00000100:00100000:0.0:1789302663.436603:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:0.0:1789302663.436613:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350234880:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302663.436681:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350234880:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302663.436688:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:0.0:1789302663.436736:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235392 00000100:00100000:0.0:1789302663.436740:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235392 00000100:00100000:0.0:1789302663.436774:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:0.0:1789302663.436781:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:2348:x1876219350235392:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302663.436827:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:2348:x1876219350235392:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 46us (361us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302663.436834:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00080000:0.0:1789302663.436956:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302663.449182:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302663.449212:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302663.453346:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235136 00000100:00100000:2.0:1789302663.453361:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235136 00000100:00100000:2.0:1789302663.453455:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235648 00000100:00100000:2.0:1789302663.453458:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235648 00000100:00100000:2.0:1789302663.453489:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 275 00000100:00100000:2.0:1789302663.453498:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350235136:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302663.453608:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350235136:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 111us (17211us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302663.453636:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 275 00000100:00100000:2.0:1789302663.453656:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 276 00000100:00100000:2.0:1789302663.453664:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350235648:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302663.453692:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350235648:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (17206us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302663.453699:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 276 00000100:00100000:3.0:1789302663.455887:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302663.455927:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235008 00000100:00100000:3.0:1789302663.455935:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235008 00000100:00100000:3.0:1789302663.455983:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:3.0:1789302663.455992:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:2348:x1876219350235008:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302663.456063:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:2348:x1876219350235008:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 72us (6890us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302663.456071:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:3.0:1789302663.456093:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235264 00000100:00100000:3.0:1789302663.456097:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235264 00000100:00100000:3.0:1789302663.456121:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235520 00000100:00100000:3.0:1789302663.456124:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235520 00000100:00100000:3.0:1789302663.456146:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:3.0:1789302663.456153:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350235264:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302663.456181:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350235264:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (6968us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302663.456195:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:3.0:1789302663.456207:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 278 00000100:00100000:3.0:1789302663.456214:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350235520:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302663.456242:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350235520:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302663.456249:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 278 00000100:00080000:1.0:1789302663.458178:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302663.458185:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302663.458187:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302663.694237:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302663.694260:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371063936:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302663.694328:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302663.694455:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371063936 00000100:00100000:1.0:1789302663.694463:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371063936 00000100:00100000:1.0:1789302663.694556:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 130 00000100:00100000:1.0:1789302663.694565:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371063936:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302663.694596:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302663.694652:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371063936:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302663.694674:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 130 00000100:00100000:2.0:1789302663.695121:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371063936:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302663.823482:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302664.206268:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371064064:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302664.206337:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302664.206408:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371064064 00000100:00100000:0.0:1789302664.206415:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371064064 00000100:00100000:0.0:1789302664.206470:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:0.0:1789302664.206478:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371064064:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302664.206545:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302664.206602:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371064064:12345-0@lo:13:osp-pre-0-0.0 Request processed in 124us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.206609:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:3.0:1789302664.208238:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371064064:0@lo:13:osp-pre-0-0.0 00000020:00000080:3.0:1789302664.260923:0:17387:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007ffd66c879d0 20000000:00000080:3.0:1789302664.260940:0:17387:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff960f3ff23800 uarg=0000000000000000 00000040:00100000:3.0:1789302664.265255:0:17387:0:(llog.c:1072:llog_reverse_process()) index 41 : idx 1 00000040:00080000:3.0:1789302664.265949:0:17387:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].42, 224 off15448 20000000:01000000:3.0:1789302664.265970:0:17387:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce020 lustre test_11c (null) (null) 00000040:00080000:3.0:1789302664.266020:0:17387:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].43, 112 off15560 00000040:00080000:3.0:1789302664.266057:0:17387:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].44, 224 off15784 00000040:00080000:3.0:1789302664.266834:0:17387:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].37, 224 off14712 20000000:01000000:3.0:1789302664.266851:0:17387:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce020 lustre test_11c (null) (null) 00000040:00080000:3.0:1789302664.266887:0:17387:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].38, 120 off14832 00000040:00080000:3.0:1789302664.266939:0:17387:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].39, 224 off15056 10000000:01000000:3.0:1789302664.266953:0:17387:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:3.0:1789302664.266972:0:17387:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.266993:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f10e7fe00/0x64fe430951849aef lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f20f expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267013:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f10e7fc00/0x64fe430951849af6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f216 expref: 12 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267034:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f01516e00/0x64fe430951849b04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe430951849afd expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267040:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f01517c00/0x64fe430951849b12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe430951849b0b expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267046:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f07ef6400/0x64fe430951849b20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe430951849b19 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267053:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f10e7e000/0x64fe430951849b2e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe430951849b27 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267058:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f10e7e400/0x64fe430951849b3c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe430951849b35 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267065:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f01516200/0x64fe430951849b4a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe430951849b43 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267147:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f01517400/0x64fe430951849b58 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe430951849b51 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267155:0:17387:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f07ef6600/0x64fe430951849b97 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f224 expref: 12 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267163:0:17387:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267204:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f07ef6600/0x64fe430951849b97 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f224 expref: 12 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267231:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960f07ef6600/0x64fe430951849b97 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f224 expref: 12 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302664.267246:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064192:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302664.267417:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f01517400/0x64fe430951849b58 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe430951849b51 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267426:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f01517400/0x64fe430951849b58 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe430951849b51 expref: 18 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302664.267436:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064320:0@lo:104: 00000100:00100000:3.0:1789302664.267477:0:17387:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302664.267616:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f01516200/0x64fe430951849b4a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe430951849b43 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.267624:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f01516200/0x64fe430951849b4a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe430951849b43 expref: 18 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302664.267632:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064448:0@lo:104: 00000100:00100000:0.0:1789302664.267764:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371064320 00000100:00100000:0.0:1789302664.267772:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371064320 00000100:00100000:0.0:1789302664.267822:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 28 00000100:00100000:0.0:1789302664.267831:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17387:x1876219371064320:12345-0@lo:104: 00010000:00010000:0.0:1789302664.267843:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.267892:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1789302664.267898:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17387:x1876219371064320:12345-0@lo:104: Request processed in 69us (421us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.267904:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 28 00010000:00010000:0.0:1789302664.267930:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302664.267936:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.267950:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:0.0:1789302664.267954:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.267963:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:0.0:1789302664.267971:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302664.267976:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:0.0:1789302664.268017:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.268024:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:0.0:1789302664.268041:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.268047:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f07ef7200/0x64fe430951849b51 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.268060:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302664.268062:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302664.268064:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302664.268156:0:17387:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302664.268176:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302664.268189:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371064576:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302664.268218:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302664.269059:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f10e7e400/0x64fe430951849b3c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe430951849b35 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302664.270495:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f10e7e400/0x64fe430951849b3c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe430951849b35 expref: 18 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302664.270600:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064704:0@lo:104: 00000100:00100000:3.0:1789302664.272008:0:17387:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302664.272566:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371064576 00000100:00100000:0.0:1789302664.272576:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371064576 00010000:00010000:0.0:1789302664.272619:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f01517400/0x64fe430951849b58 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b51 expref: 19 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.272675:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1789302664.272683:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371064576:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302664.272698:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302664.272704:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f01517400/0x64fe430951849b58 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b51 expref: 19 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.272716:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f01517400/0x64fe430951849b58 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe430951849b51 expref: 19 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.272734:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302664.272785:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371064576:12345-0@lo:103:ldlm_bl.0 Request processed in 103us (4565us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.272791:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:1.0:1789302664.275442:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371064448 00000100:00100000:1.0:1789302664.275455:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371064448 00000100:00100000:1.0:1789302664.275493:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371064704 00000100:00100000:1.0:1789302664.275496:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371064704 00000100:00100000:1.0:1789302664.275542:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 29 00000100:00100000:1.0:1789302664.275551:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17387:x1876219371064448:12345-0@lo:104: 00010000:00010000:1.0:1789302664.275568:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.275626:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1789302664.275632:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17387:x1876219371064448:12345-0@lo:104: Request processed in 83us (7476us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302664.275639:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 29 00000100:00100000:1.0:1789302664.275664:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30 00000100:00100000:1.0:1789302664.275670:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17387:x1876219371064704:12345-0@lo:104: 00010000:00010000:1.0:1789302664.275677:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.275705:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:1.0:1789302664.275710:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17387:x1876219371064704:12345-0@lo:104: Request processed in 41us (3715us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302664.275715:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30 00000100:00100000:1.0:1789302664.275759:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371064576:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789302664.275784:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1789302664.275789:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.275813:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302664.275817:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.275826:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302664.275833:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302664.275839:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789302664.276467:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f10e7e000/0x64fe430951849b2e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe430951849b27 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.276482:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f10e7e000/0x64fe430951849b2e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe430951849b27 expref: 18 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.276497:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064832:0@lo:104: 00000100:00100000:2.0:1789302664.277098:0:17387:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789302664.277143:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f07ef6400/0x64fe430951849b20 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe430951849b19 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.277152:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f07ef6400/0x64fe430951849b20 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe430951849b19 expref: 18 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.277162:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064960:0@lo:104: 00000100:00100000:2.0:1789302664.277194:0:17387:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789302664.277212:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f01517c00/0x64fe430951849b12 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe430951849b0b expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.277220:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f01517c00/0x64fe430951849b12 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe430951849b0b expref: 18 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.277228:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065088:0@lo:104: 00000100:00100000:2.0:1789302664.277247:0:17387:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789302664.277907:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f01516e00/0x64fe430951849b04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe430951849afd expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.277918:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f01516e00/0x64fe430951849b04 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe430951849afd expref: 18 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.277928:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065216:0@lo:104: 00000100:00100000:2.0:1789302664.277966:0:17387:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789302664.277990:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f10e7fc00/0x64fe430951849af6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f216 expref: 12 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.278009:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960f10e7fc00/0x64fe430951849af6 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f216 expref: 12 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.278019:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ae300 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065344:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302664.278090:0:17387:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f10e7fe00/0x64fe430951849aef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f20f expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.278099:0:17387:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960f10e7fe00/0x64fe430951849aef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f20f expref: 12 pid: 5838 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.278123:0:17387:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065472:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302664.278148:0:17387:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302664.278171:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064192:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302664.278184:0:17387:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f01517400/0x64fe430951849b58 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe430951849b51 expref: 18 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.278198:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064320:0@lo:104: 00000100:00100000:2.0:1789302664.278208:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064448:0@lo:104: 00000100:00100000:2.0:1789302664.278216:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064704:0@lo:104: 00000100:00100000:2.0:1789302664.278717:0:5827:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:2.0:1789302664.278732:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371064832 00000100:00100000:2.0:1789302664.278738:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371064832 00000100:00100000:2.0:1789302664.278775:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371064960 00000100:00100000:2.0:1789302664.278778:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371064960 00000100:00100000:2.0:1789302664.278790:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371065088 00000100:00100000:2.0:1789302664.278793:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371065088 00000100:00100000:2.0:1789302664.278805:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371065216 00000100:00100000:2.0:1789302664.278808:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371065216 00000100:00100000:2.0:1789302664.278823:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 31 00000100:00100000:2.0:1789302664.278830:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17387:x1876219371064832:12345-0@lo:104: 00010000:00010000:2.0:1789302664.278843:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.278896:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:2.0:1789302664.278903:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17387:x1876219371064832:12345-0@lo:104: Request processed in 73us (1806us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302664.278909:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 31 00000100:00100000:2.0:1789302664.278936:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 32 00000100:00100000:2.0:1789302664.278941:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17387:x1876219371064960:12345-0@lo:104: 00010000:00010000:2.0:1789302664.278949:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.278977:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:2.0:1789302664.278981:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17387:x1876219371064960:12345-0@lo:104: Request processed in 41us (1787us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302664.278986:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 32 00000100:00100000:2.0:1789302664.279010:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 33 00000100:00100000:2.0:1789302664.279016:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17387:x1876219371065088:12345-0@lo:104: 00010000:00010000:2.0:1789302664.279023:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.279048:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to priority blp list, 4 blwis in pool 00000100:00100000:2.0:1789302664.279053:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17387:x1876219371065088:12345-0@lo:104: Request processed in 38us (1806us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302664.279058:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 33 00000100:00100000:2.0:1789302664.279083:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789302664.279089:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f750850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17387:x1876219371065216:12345-0@lo:104: 00010000:00010000:2.0:1789302664.279097:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.279123:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to priority blp list, 5 blwis in pool 00000100:00100000:2.0:1789302664.279127:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f750850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17387:x1876219371065216:12345-0@lo:104: Request processed in 40us (1161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302664.279132:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:2.0:1789302664.279194:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064832:0@lo:104: 00000100:00100000:2.0:1789302664.279203:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371064960:0@lo:104: 00000100:00100000:2.0:1789302664.279210:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065088:0@lo:104: 00000100:00100000:2.0:1789302664.279216:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065216:0@lo:104: 00010000:00010000:1.0:1789302664.279741:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279750:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302664.279778:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279785:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f07ef6a00/0x64fe430951849b43 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279799:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1789302664.279804:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279810:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7800 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302664.279814:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279836:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302664.279843:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302664.279849:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302664.279863:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279868:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302664.279873:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279879:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f07ef7800/0x64fe430951849b35 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279888:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1789302664.279891:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279896:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f73c00 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302664.279899:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279905:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302664.279910:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302664.279920:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302664.279931:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279937:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302664.279951:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279958:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f25f73c00/0x64fe430951849b27 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b2e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279967:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1789302664.279970:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279975:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72000 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302664.279978:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.279984:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302664.279990:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302664.280008:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302664.280019:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.280025:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302664.280037:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.280043:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f25f72000/0x64fe430951849b19 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b20 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.280052:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1789302664.280055:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.280060:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7f800 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302664.280063:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.280145:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371065600:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302664.280190:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302664.280226:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789302664.280232:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371065728:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302664.280252:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302664.280359:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371065600 00000100:00100000:0.0:1789302664.280365:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371065600 00010000:00010000:0.0:1789302664.280388:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f01516200/0x64fe430951849b4a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b43 expref: 18 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.280411:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371065728 00000100:00100000:0.0:1789302664.280415:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371065728 00010000:00010000:0.0:1789302664.280426:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f10e7e400/0x64fe430951849b3c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b35 expref: 19 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.280451:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1789302664.280457:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371065600:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302664.280468:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302664.280473:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f01516200/0x64fe430951849b4a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b43 expref: 19 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.280481:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f01516200/0x64fe430951849b4a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe430951849b43 expref: 19 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.280496:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f01516200/0x64fe430951849b4a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe430951849b43 expref: 19 pid: 5837 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.280637:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302664.280679:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:4acabb58-ab41-4250-a3d6-ebb667f59134+18:3302:x1876219371065600:12345-0@lo:103:ldlm_bl.0 Request processed in 223us (489us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.280685:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1789302664.280702:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1789302664.280708:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3302:x1876219371065728:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302664.280715:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302664.280719:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f10e7e400/0x64fe430951849b3c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b35 expref: 17 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.280728:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f10e7e400/0x64fe430951849b3c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe430951849b35 expref: 17 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.280738:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f10e7e400/0x64fe430951849b3c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe430951849b35 expref: 17 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.280758:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302664.280781:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:4acabb58-ab41-4250-a3d6-ebb667f59134+16:3302:x1876219371065728:12345-0@lo:103:ldlm_bl.0 Request processed in 73us (528us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.280787:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1789302664.280824:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371065728:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302664.280830:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371065600:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789302664.280947:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302664.280954:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302664.280958:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302664.280977:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.280982:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302664.281011:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281017:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f10e7f800/0x64fe430951849b0b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b12 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281026:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302664.281036:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281042:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72400 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302664.281045:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281051:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302664.281057:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302664.281062:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302664.281084:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281090:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302664.281097:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281102:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f25f72400/0x64fe430951849afd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe430951849b04 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281115:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302664.281117:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302664.281118:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302664.281238:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371065984:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302664.281278:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302664.281307:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302664.281312:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371065856:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302664.281342:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302664.281367:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371065984 00000100:00100000:1.0:1789302664.281371:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371065984 00010000:00010000:1.0:1789302664.281387:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f07ef6400/0x64fe430951849b20 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b19 expref: 16 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302664.281403:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371065856 00000100:00100000:1.0:1789302664.281406:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371065856 00010000:00010000:1.0:1789302664.281417:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f10e7e000/0x64fe430951849b2e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b27 expref: 17 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302664.281431:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789302664.281436:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3301:x1876219371065984:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789302664.281444:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302664.281447:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f07ef6400/0x64fe430951849b20 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b19 expref: 17 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281455:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f07ef6400/0x64fe430951849b20 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe430951849b19 expref: 17 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281463:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f07ef6400/0x64fe430951849b20 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe430951849b19 expref: 17 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281471:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302664.281497:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+16:3301:x1876219371065984:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (219us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302664.281719:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789302664.281734:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:1.0:1789302664.281741:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:3301:x1876219371065856:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789302664.281758:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302664.281762:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f10e7e000/0x64fe430951849b2e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b27 expref: 15 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281771:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f10e7e000/0x64fe430951849b2e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe430951849b27 expref: 15 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281781:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f10e7e000/0x64fe430951849b2e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe430951849b27 expref: 15 pid: 5838 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.281789:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302664.281812:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:3301:x1876219371065856:12345-0@lo:103:ldlm_bl.0 Request processed in 72us (470us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302664.281818:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:1.0:1789302664.281875:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371065856:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302664.281882:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371065984:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789302664.284961:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302664.285386:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371066240:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302664.285454:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302664.285530:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302664.285536:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371066112:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302664.285558:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302664.285590:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371066240 00000100:00100000:0.0:1789302664.285597:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371066240 00010000:00010000:0.0:1789302664.285626:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f01516e00/0x64fe430951849b04 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849afd expref: 14 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.285662:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371066112 00000100:00100000:0.0:1789302664.285665:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371066112 00010000:00010000:0.0:1789302664.285677:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f01517c00/0x64fe430951849b12 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b0b expref: 15 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.285693:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:0.0:1789302664.285701:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:3303:x1876219371066240:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302664.285713:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302664.285718:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f01516e00/0x64fe430951849b04 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849afd expref: 15 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.285746:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f01516e00/0x64fe430951849b04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe430951849afd expref: 15 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.285763:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f01516e00/0x64fe430951849b04 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe430951849afd expref: 15 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.285777:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302664.285812:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:3303:x1876219371066240:12345-0@lo:103:ldlm_bl.0 Request processed in 112us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.285819:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:0.0:1789302664.285840:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1789302664.285846:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:3303:x1876219371066112:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302664.285853:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302664.285857:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f01517c00/0x64fe430951849b12 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe430951849b0b expref: 13 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.285865:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f01517c00/0x64fe430951849b12 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe430951849b0b expref: 13 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.285874:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f01517c00/0x64fe430951849b12 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe430951849b0b expref: 13 pid: 5836 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.285884:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302664.285906:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+12:3303:x1876219371066112:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.285911:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:0.0:1789302664.285952:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371066112:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302664.285960:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371066240:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302664.286438:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302664.286524:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302664.287664:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235776 00000100:00100000:0.0:1789302664.287674:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235776 00010000:00010000:0.0:1789302664.287712:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f07ef6600/0x64fe430951849b97 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f224 expref: 13 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.287739:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350235904 00000100:00100000:0.0:1789302664.287742:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350235904 00010000:00010000:0.0:1789302664.287755:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f10e7fc00/0x64fe430951849af6 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f216 expref: 14 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302664.287773:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 209 00000100:00100000:0.0:1789302664.287782:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:2348:x1876219350235776:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302664.287794:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302664.287799:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f07ef6600/0x64fe430951849b97 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f224 expref: 14 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.287810:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f07ef6600/0x64fe430951849b97 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f224 expref: 14 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.287826:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f07ef6600/0x64fe430951849b97 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f224 expref: 14 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.287841:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302664.287893:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350235776:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 112us (1458us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.287901:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 209 00000100:00100000:0.0:1789302664.287920:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 210 00000100:00100000:0.0:1789302664.287927:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2348:x1876219350235904:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302664.287952:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302664.287957:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f10e7fc00/0x64fe430951849af6 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f216 expref: 12 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.287965:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f10e7fc00/0x64fe430951849af6 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f216 expref: 12 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302664.287975:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302664.287993:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2348:x1876219350235904:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 67us (1492us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302664.288011:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 210 00000100:00100000:1.0:1789302664.288312:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302664.288338:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236032 00000100:00100000:1.0:1789302664.288343:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236032 00010000:00010000:1.0:1789302664.288361:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f10e7fe00/0x64fe430951849aef lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f20f expref: 12 pid: 5838 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302664.288380:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 211 00000100:00100000:1.0:1789302664.288386:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2348:x1876219350236032:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302664.288393:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302664.288404:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f10e7fe00/0x64fe430951849aef lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f20f expref: 12 pid: 5838 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288414:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f10e7fe00/0x64fe430951849aef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f20f expref: 12 pid: 5838 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288427:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50290000000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288436:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50290000000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288452:0:5830:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288459:0:5830:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288467:0:5830:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288474:0:5830:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288480:0:5830:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302664.288485:0:5830:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288495:0:5830:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288532:0:5830:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.288549:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302664.288570:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2348:x1876219350236032:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 184us (260us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302664.288576:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 211 00010000:00010000:3.0:1789302664.289717:0:17387:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f10e7fc00/0x64fe430951849af6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f216 expref: 11 pid: 5837 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302664.289733:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ae300 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065344:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302664.289792:0:17387:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f10e7fe00/0x64fe430951849aef lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f20f expref: 10 pid: 5838 timeout: 601 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302664.289802:0:17387:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17387:1876219371065472:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302664.289849:0:17387:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.289858:0:17387:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.289865:0:17387:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302664.289872:0:17387:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e03c2d600/0x64fe43095184c9e2 lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17387 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302664.844726:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:3.0:1789302665.285682:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 10 batchid updated 00000020:00080000:2.0:1789302665.285719:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 10 00000020:00080000:2.0:1789302665.285733:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 10 00000001:00080000:3.0:1789302665.285875:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967775 is committed 00010000:00010000:3.0:1789302665.868850:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302666.004156:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302666.004296:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236160 00000100:00100000:0.0:1789302666.004305:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236160 00000100:00100000:0.0:1789302666.004359:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 358 00000100:00100000:0.0:1789302666.004369:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+10:5588:x1876219350236160:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302666.004384:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302666.004403:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffc2 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.004439:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 11 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.004451:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 11 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.004460:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3000, rc 0) 00000100:00100000:0.0:1789302666.004543:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350236160:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 176us (397us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.004563:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 358 00000100:00100000:2.0:1789302666.008002:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302666.008090:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236288 00000100:00100000:0.0:1789302666.008100:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236288 00000100:00100000:0.0:1789302666.008164:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 359 00000100:00100000:0.0:1789302666.008173:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350236288:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.008432:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350236288:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 259us (446us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.008440:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 359 00000100:00100000:1.0:1789302666.026313:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302666.028527:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236416 00000100:00100000:2.0:1789302666.028540:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236416 00000100:00100000:2.0:1789302666.028608:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 360 00000100:00100000:2.0:1789302666.028618:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350236416:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.029201:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350236416:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 582us (2895us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302666.029216:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 360 00000100:00100000:3.0:1789302666.033682:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302666.034852:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236544 00000100:00100000:2.0:1789302666.034863:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236544 00000100:00100000:2.0:1789302666.034958:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 361 00000100:00100000:2.0:1789302666.034968:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22440:x1876219350236544:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302666.035551:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22440:x1876219350236544:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 582us (1875us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302666.035560:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 361 00000100:00100000:1.0:1789302666.038792:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302666.038897:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236672 00000100:00100000:2.0:1789302666.038907:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236672 00000100:00100000:2.0:1789302666.039004:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:2.0:1789302666.039013:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22440:x1876219350236672:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302666.039452:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22440:x1876219350236672:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 438us (667us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302666.039460:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:1.0:1789302666.044918:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302666.052845:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236800 00000100:00100000:0.0:1789302666.052859:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236800 00000100:00100000:0.0:1789302666.052920:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 363 00000100:00100000:0.0:1789302666.052930:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350236800:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302666.052945:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302666.052962:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffc9 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.053000:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.053011:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.053020:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3c00, rc 0) 00000100:00100000:0.0:1789302666.055516:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350236800:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 2566us (10589us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.055528:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 363 00000100:00100000:0.0:1789302666.069531:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302666.069612:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350236928 00000100:00100000:1.0:1789302666.069621:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350236928 00000100:00100000:1.0:1789302666.069672:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 364 00000100:00100000:1.0:1789302666.069681:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350236928:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.069958:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350236928:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 277us (466us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.069966:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 364 00000100:00100000:1.0:1789302666.075606:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302666.075685:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237056 00000100:00100000:3.0:1789302666.075695:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237056 00000100:00100000:3.0:1789302666.075745:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 365 00000100:00100000:3.0:1789302666.075755:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350237056:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.076203:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350237056:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 449us (604us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.076212:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 365 00000100:00100000:3.0:1789302666.079128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302666.079584:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237184 00000100:00100000:1.0:1789302666.079594:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237184 00000100:00100000:1.0:1789302666.079696:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:1.0:1789302666.079706:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22441:x1876219350237184:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:1.0:1789302666.079963:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22441:x1876219350237184:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 257us (841us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.079970:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:1.0:1789302666.083973:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302666.086669:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237312 00000100:00100000:3.0:1789302666.086683:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237312 00000100:00100000:3.0:1789302666.086736:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 367 00000100:00100000:3.0:1789302666.086746:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22441:x1876219350237312:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:3.0:1789302666.087110:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22441:x1876219350237312:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 364us (3144us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.087118:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 367 00000100:00100000:2.0:1789302666.091072:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302666.091566:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237440 00000100:00100000:1.0:1789302666.091575:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237440 00000100:00100000:1.0:1789302666.091627:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 368 00000100:00100000:1.0:1789302666.091636:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350237440:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302666.091748:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302666.091765:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffd0 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.091793:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.091805:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.091813:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eba00, rc 0) 00000100:00100000:1.0:1789302666.091874:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350237440:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 239us (812us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.091881:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 368 00000100:00100000:1.0:1789302666.094843:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302666.094934:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237568 00000100:00100000:0.0:1789302666.094944:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237568 00000100:00100000:0.0:1789302666.095002:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 369 00000100:00100000:0.0:1789302666.095011:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350237568:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.095603:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350237568:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 590us (765us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302666.095612:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 369 10000000:01000000:3.0:1789302666.322596:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1789302666.322606:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f120ab000 from 42 10000000:01000000:3.0:1789302666.322612:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1789302666.322686:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302666.322698:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302666.322709:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302666.322736:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371066368:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.322799:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302666.322867:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.323083:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371066368 00000100:00100000:1.0:1789302666.323091:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371066368 00000100:00100000:1.0:1789302666.323141:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:1.0:1789302666.323149:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+12:5841:x1876219371066368:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302666.323163:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302666.323175:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184c9fe expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.323199:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184c9fe expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.323217:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184c9fe expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.323227:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb200, rc 0) 00000100:00100000:1.0:1789302666.323281:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371066368:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 133us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.323288:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789302666.323415:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371066368:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302666.323430:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.323439:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302666.323491:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371066496:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.323554:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.323571:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302666.323623:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371066496 00000100:00100000:0.0:1789302666.323628:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371066496 00000100:00100000:0.0:1789302666.323665:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:0.0:1789302666.323672:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371066496:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.323897:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371066496:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 225us (341us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.323904:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1789302666.324044:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371066496:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.324071:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371066624:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.324107:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.324127:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302666.324171:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371066624 00000100:00100000:0.0:1789302666.324175:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371066624 00000100:00100000:0.0:1789302666.324191:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:0.0:1789302666.324197:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371066624:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.324351:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371066624:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 153us (243us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.324358:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:2.0:1789302666.324659:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371066624:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.325073:0:17388:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17388:1876219371066752:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.325112:0:17388:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.325142:0:17388:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302666.325489:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371066752 00000100:00100000:0.0:1789302666.325495:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371066752 00000100:00100000:0.0:1789302666.325551:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:0.0:1789302666.325558:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17388:x1876219371066752:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302666.325775:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17388:x1876219371066752:12345-0@lo:502:llog_process_th.0 Request processed in 216us (661us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.325781:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789302666.325924:0:17388:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17388:1876219371066752:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.325965:0:17388:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17388:1876219371066880:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.326010:0:17388:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.326036:0:17388:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.326094:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371066880 00000100:00100000:1.0:1789302666.326098:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371066880 00000100:00100000:1.0:1789302666.326118:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:1.0:1789302666.326123:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17388:x1876219371066880:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302666.326267:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17388:x1876219371066880:12345-0@lo:502:llog_process_th.0 Request processed in 132us (245us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.326273:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1789302666.326403:0:17388:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17388:1876219371066880:0@lo:502:llog_process_th.0 00000040:00080000:2.0:1789302666.326421:0:17388:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 45 count 45 00000020:01000000:0.0:1789302666.328601:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 42-44 (rc=0) 10000000:01000000:0.0:1789302666.328630:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1789302666.328651:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.328664:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302666.328673:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:0.0:1789302666.328675:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:0.0:1789302666.328678:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:0.0:1789302666.328713:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.328721:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.328731:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302666.328752:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067008:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.328807:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.328836:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b000 going to sleep for 11 seconds 00000100:00100000:0.0:1789302666.328850:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067008 00000100:00100000:0.0:1789302666.328855:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067008 00000100:00100000:0.0:1789302666.328894:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:0.0:1789302666.328915:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371067008:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302666.328929:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302666.328941:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca0c expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.328964:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca0c expref: 14 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.328975:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca0c expref: 14 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.328993:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:0.0:1789302666.329031:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371067008:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 117us (224us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.329037:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1789302666.329248:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067008:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302666.329264:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.329273:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302666.329329:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067136:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.329363:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.329373:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.329462:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067136 00000100:00100000:1.0:1789302666.329467:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067136 00000100:00100000:1.0:1789302666.329495:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1789302666.329525:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371067136:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.329753:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371067136:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 251us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.329760:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:2.0:1789302666.331409:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067136:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.331459:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067264:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.331534:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.331573:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.332952:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067264 00000100:00100000:1.0:1789302666.332961:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067264 00000100:00100000:1.0:1789302666.333023:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:1.0:1789302666.333032:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371067264:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.333407:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371067264:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 376us (1871us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.333415:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1789302666.333552:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067264:0@lo:503:ll_cfg_requeue.0 00000040:00080000:1.0:1789302666.337263:0:17389:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:0.0:1789302666.337719:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:0.0:1789302666.337742:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:0.0:1789302666.337753:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.337765:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302666.337775:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1789302666.337778:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f1d5a8000 from 42 10000000:01000000:0.0:1789302666.337781:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1789302666.337834:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.337843:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.337859:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302666.337882:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067392:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.337942:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.337996:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.339317:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067392 00000100:00100000:1.0:1789302666.339328:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067392 00000100:00100000:1.0:1789302666.339384:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 378 00000100:00100000:1.0:1789302666.339393:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371067392:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302666.339407:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302666.339422:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca1a expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.339444:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca1a expref: 15 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.339455:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca1a expref: 15 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.339463:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebe00, rc 0) 00000100:00100000:1.0:1789302666.339532:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371067392:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 140us (1589us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.339539:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 378 00000100:00100000:0.0:1789302666.339851:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067392:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302666.339871:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.339880:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302666.339946:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067520:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.339974:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.340015:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.340067:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067520 00000100:00100000:3.0:1789302666.340071:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067520 00000100:00100000:3.0:1789302666.340103:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:3.0:1789302666.340110:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371067520:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.340362:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371067520:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 251us (385us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.340369:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00100000:0.0:1789302666.340557:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067520:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.340585:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067648:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.340622:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.340656:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.341495:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067648 00000100:00100000:3.0:1789302666.341536:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067648 00000100:00100000:3.0:1789302666.341572:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:3.0:1789302666.341579:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371067648:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.341761:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371067648:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 181us (1136us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.341767:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:0.0:1789302666.341925:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371067648:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.344972:0:17390:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17390:1876219371067776:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302666.345059:0:17390:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302666.345100:0:17390:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80600 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.345589:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067776 00000100:00100000:3.0:1789302666.345597:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067776 00000100:00100000:3.0:1789302666.345652:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:3.0:1789302666.345660:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17390:x1876219371067776:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302666.345901:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17390:x1876219371067776:12345-0@lo:502:llog_process_th.0 Request processed in 241us (841us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.345909:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:1.0:1789302666.346069:0:17390:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17390:1876219371067776:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302666.346117:0:17390:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17390:1876219371067904:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302666.346155:0:17390:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302666.346186:0:17390:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80600 going to sleep for 11 seconds 00000100:00100000:2.0:1789302666.346226:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371067904 00000100:00100000:2.0:1789302666.346231:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371067904 00000100:00100000:2.0:1789302666.346266:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1789302666.346272:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17390:x1876219371067904:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.346422:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17390:x1876219371067904:12345-0@lo:502:llog_process_th.0 Request processed in 149us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302666.346429:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:1.0:1789302666.346621:0:17390:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17390:1876219371067904:0@lo:502:llog_process_th.0 00000040:00080000:1.0:1789302666.346649:0:17390:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 45 count 45 00000020:01000000:2.0:1789302666.346964:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 42-44 (rc=0) 10000000:01000000:2.0:1789302666.346995:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1789302666.347004:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.347016:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302666.347024:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:2.0:1789302666.347027:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:2.0:1789302666.347029:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:2.0:1789302666.347053:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.347068:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.347079:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302666.347094:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068032:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.347124:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.347155:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.347219:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068032 00000100:00100000:1.0:1789302666.347225:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068032 00000100:00100000:1.0:1789302666.347257:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 383 00000100:00100000:1.0:1789302666.347264:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371068032:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302666.347273:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302666.347284:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca28 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.347317:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca28 expref: 16 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.347328:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca28 expref: 16 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.347336:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb000, rc 0) 00000100:00100000:1.0:1789302666.347371:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371068032:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 108us (248us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.347377:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 383 00000100:00100000:2.0:1789302666.347574:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068032:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302666.347586:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.347596:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302666.347646:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068160:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.347671:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.347685:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.347788:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068160 00000100:00100000:3.0:1789302666.347793:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068160 00000100:00100000:3.0:1789302666.347817:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1789302666.347823:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371068160:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.348060:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371068160:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 236us (387us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.348067:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:2.0:1789302666.349893:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068160:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.349946:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068288:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.350019:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.350047:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302666.350112:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068288 00000100:00100000:0.0:1789302666.350118:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068288 00000100:00100000:0.0:1789302666.350161:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:0.0:1789302666.350169:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371068288:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.350958:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371068288:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 787us (936us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.351565:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:2.0:1789302666.351761:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068288:0@lo:503:ll_cfg_requeue.0 00000040:00080000:2.0:1789302666.353947:0:17391:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000020:01000000:1.0:1789302666.354253:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:1.0:1789302666.354271:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:1.0:1789302666.354282:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.354294:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302666.354320:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1789302666.354323:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f0511c000 from 42 10000000:01000000:1.0:1789302666.354326:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1789302666.354360:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.354368:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.354378:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302666.354400:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068416:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.354454:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302666.354491:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80f00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302666.357586:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068416 00000100:00100000:2.0:1789302666.357599:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068416 00000100:00100000:2.0:1789302666.357813:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:2.0:1789302666.357824:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371068416:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302666.357839:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302666.357856:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca36 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.357882:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca36 expref: 17 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.357894:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca36 expref: 17 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.357902:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe800, rc 0) 00000100:00100000:2.0:1789302666.358019:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371068416:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 197us (3564us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302666.358026:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1789302666.373195:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068416:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302666.373224:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.373237:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302666.373311:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068544:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.373363:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302666.373404:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.373494:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068544 00000100:00100000:3.0:1789302666.373519:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068544 00000100:00100000:3.0:1789302666.373553:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:3.0:1789302666.373561:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371068544:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.373808:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371068544:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 246us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.373815:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:0.0:1789302666.374023:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068544:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.374052:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068672:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.374089:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.374116:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.374906:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068672 00000100:00100000:3.0:1789302666.374914:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068672 00000100:00100000:3.0:1789302666.374956:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:3.0:1789302666.374963:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371068672:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.375176:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371068672:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 212us (1085us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.375183:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00100000:2.0:1789302666.380450:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371068672:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.381035:0:17392:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17392:1876219371068800:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.381100:0:17392:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.381140:0:17392:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25bed540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.381189:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068800 00000100:00100000:3.0:1789302666.381197:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068800 00000100:00100000:3.0:1789302666.381239:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:3.0:1789302666.381249:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17392:x1876219371068800:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302666.381488:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17392:x1876219371068800:12345-0@lo:502:llog_process_th.0 Request processed in 239us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.381496:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789302666.381761:0:17392:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17392:1876219371068800:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.381801:0:17392:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17392:1876219371068928:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.381831:0:17392:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.381857:0:17392:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25bed540 going to sleep for 11 seconds 00000100:00100000:0.0:1789302666.381899:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371068928 00000100:00100000:0.0:1789302666.381903:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371068928 00000100:00100000:0.0:1789302666.381931:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 390 00000100:00100000:0.0:1789302666.381937:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17392:x1876219371068928:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302666.382090:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17392:x1876219371068928:12345-0@lo:502:llog_process_th.0 Request processed in 152us (257us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.382097:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 390 00000100:00100000:2.0:1789302666.382157:0:17392:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17392:1876219371068928:0@lo:502:llog_process_th.0 00000040:00080000:2.0:1789302666.382175:0:17392:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 45 count 45 00000020:01000000:0.0:1789302666.382308:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 42-44 (rc=0) 10000000:01000000:0.0:1789302666.382328:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1789302666.382337:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.382348:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302666.382358:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:0.0:1789302666.382360:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 37 10000000:01000000:0.0:1789302666.382363:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:0.0:1789302666.382385:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.382399:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.382408:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302666.382421:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069056:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.382445:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.382467:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b000 going to sleep for 11 seconds 00000100:00100000:2.0:1789302666.382619:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069056 00000100:00100000:2.0:1789302666.382624:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069056 00000100:00100000:2.0:1789302666.382654:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 391 00000100:00100000:2.0:1789302666.382660:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371069056:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302666.382669:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302666.382681:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca44 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.382701:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca44 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.382712:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca44 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.382720:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf800, rc 0) 00000100:00100000:2.0:1789302666.382746:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371069056:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 86us (300us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302666.382752:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 391 00000100:00100000:0.0:1789302666.382875:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069056:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302666.382887:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302666.382896:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302666.382936:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069184:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.382958:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.382994:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b000 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.383048:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069184 00000100:00100000:3.0:1789302666.383053:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069184 00000100:00100000:3.0:1789302666.383856:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789302666.383867:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371069184:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.384119:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371069184:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 254us (1159us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.384126:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:1.0:1789302666.384212:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069184:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.384249:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069312:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.384290:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302666.388257:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302666.388952:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069312 00000100:00100000:0.0:1789302666.388962:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069312 00000100:00100000:0.0:1789302666.389019:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:0.0:1789302666.389028:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371069312:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.389441:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371069312:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 413us (5149us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.389449:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1789302666.389597:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069312:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302666.390114:0:17393:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17393:1876219371069440:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302666.390160:0:17393:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302666.390192:0:17393:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e840 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.390228:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069440 00000100:00100000:1.0:1789302666.390234:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069440 00000100:00100000:1.0:1789302666.390273:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789302666.390280:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17393:x1876219371069440:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302666.391060:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17393:x1876219371069440:12345-0@lo:502:llog_process_th.0 Request processed in 778us (897us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302666.391068:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1789302666.391149:0:17393:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17393:1876219371069440:0@lo:502:llog_process_th.0 00000020:01000000:1.0:1789302666.391176:0:17393:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1789302666.391186:0:17393:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1789302666.391188:0:17393:0:(obd_config.c:1563:class_process_config()) marker 29 (0x1) lustre-MDT0000-m new lustre.test_11c 00000020:00000080:1.0:1789302666.391193:0:17393:0:(obd_config.c:1472:class_process_config()) processing cmd: ce020 00020000:01000000:1.0:1789302666.391244:0:17393:0:(lod_pool.c:475:lod_pool_new()) test_11c is pool #1 00080000:10000000:3.0:1789302666.399449:0:17393:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x37:0x0] (8/8/4) registered 00080000:10000000:0.0:1789302666.444192:0:17393:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x38:0x0] (8/8/4) registered 00080000:10000000:1.0:1789302666.531647:0:17393:0:(osd_index.c:2103:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x39:0x0] (8/8/4) registered 00000100:00100000:2.0:1789302666.585245:0:17393:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17393:1876219371069568:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302666.585331:0:17393:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.585387:0:17393:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0dcbe780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302666.585589:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069568 00000100:00100000:3.0:1789302666.585597:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069568 00000100:00100000:3.0:1789302666.585649:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789302666.585657:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17393:x1876219371069568:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302666.585860:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17393:x1876219371069568:12345-0@lo:502:llog_process_th.0 Request processed in 204us (529us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302666.585868:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:2.0:1789302666.586166:0:17393:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17393:1876219371069568:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1789302666.586195:0:17393:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:2.0:1789302666.586202:0:17393:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1789302666.586204:0:17393:0:(obd_config.c:1563:class_process_config()) marker 29 (0x2) lustre-MDT0000-m new lustre.test_11c 00000040:00080000:2.0:1789302666.586215:0:17393:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 40 count 40 00000020:01000000:3.0:1789302666.593956:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 37-39 (rc=0) 10000000:01000000:3.0:1789302666.594001:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:3.0:1789302666.594013:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302666.594048:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302666.594071:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:3.0:1789302666.594074:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff960f050d19c0 from 1 10000000:01000000:3.0:1789302666.594078:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:3.0:1789302666.594117:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302666.594130:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302666.594141:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302666.594176:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069696:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302666.594245:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302666.594304:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33411300 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.594480:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069696 00000100:00100000:1.0:1789302666.594489:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069696 00000100:00100000:1.0:1789302666.594569:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:1.0:1789302666.594579:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371069696:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302666.594592:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302666.594605:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca52 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.594638:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca52 expref: 19 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.594649:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca52 expref: 19 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.594668:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:1.0:1789302666.594712:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371069696:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 135us (467us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302666.594719:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1789302666.594892:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069696:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302666.594908:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.594917:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302666.594974:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069824:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302666.595014:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302666.595034:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0dcbe780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302666.595401:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069824 00000100:00100000:1.0:1789302666.595407:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069824 00000100:00100000:1.0:1789302666.595446:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789302666.595453:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371069824:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302666.595721:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371069824:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 266us (704us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302666.595728:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:2.0:1789302666.596476:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aed80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371069824:0@lo:501:ll_cfg_requeue.0 10000000:01000000:2.0:1789302666.596555:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:2.0:1789302666.596564:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302666.596573:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302666.892657:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302667.922248:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302668.300862:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371070208:0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.300986:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302668.301085:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302668.301092:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371070336:0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.301121:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302668.301165:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070208 00000100:00100000:3.0:1789302668.301176:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070208 00000100:00100000:3.0:1789302668.301492:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1789302668.301533:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371070208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.301617:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371070208:12345-0@lo:400:kworker.0 Request processed in 115us (632us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.301626:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1789302668.301665:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070336 00000100:00100000:3.0:1789302668.301669:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070336 00000100:00100000:3.0:1789302668.301708:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:3.0:1789302668.301715:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371070336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.301746:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371070336:12345-0@lo:400:kworker.0 Request processed in 32us (625us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.301752:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00080000:3.0:1789302668.301830:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302668.301835:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302668.302190:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371070336:0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.302201:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371070208:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.302369:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371070080:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.302424:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.302469:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371070592:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.302521:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.302555:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789302668.302561:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371069952:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.302602:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.302637:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371070464:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.302661:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302668.303311:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070080 00000100:00100000:3.0:1789302668.303329:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070080 00000100:00100000:3.0:1789302668.303377:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070464 00000100:00100000:3.0:1789302668.303380:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070464 00000100:00100000:3.0:1789302668.303414:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:3.0:1789302668.303422:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371070080:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.303492:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371070080:12345-0@lo:400:kworker.0 Request processed in 70us (1067us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.303498:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:3.0:1789302668.303538:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:3.0:1789302668.303544:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371070464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.303577:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371070464:12345-0@lo:400:kworker.0 Request processed in 29us (911us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.303592:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00100000:3.0:1789302668.303615:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070592 00000100:00100000:3.0:1789302668.303619:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070592 00000100:00100000:3.0:1789302668.303660:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:3.0:1789302668.303667:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371070592:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302668.303713:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371070592:12345-0@lo:400:kworker.0 Request processed in 46us (1213us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.303719:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:1.0:1789302668.304780:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371069952 00000100:00100000:1.0:1789302668.304791:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371069952 00000100:00100000:1.0:1789302668.304849:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789302668.304868:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371069952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302668.304932:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371069952:12345-0@lo:400:kworker.0 Request processed in 74us (2328us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302668.304940:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1789302668.312180:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371069952:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.312216:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371070464:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.312244:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371070080:0@lo:400:kworker.0 00000100:00100000:2.0:1789302668.312265:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371070592:0@lo:400:kworker.0 00000100:00080000:3.0:1789302668.314616:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302668.314630:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302668.556447:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.556492:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.556558:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.556581:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.556612:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.556847:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.556876:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.556910:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237952 00000100:00100000:3.0:1789302668.556922:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237952 00000100:00100000:3.0:1789302668.556999:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238464 00000100:00100000:3.0:1789302668.557002:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238464 00000100:00100000:3.0:1789302668.557021:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00100000:3.0:1789302668.557032:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350237952:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302668.557912:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237696 00000100:00100000:2.0:1789302668.557922:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237696 00000100:00100000:2.0:1789302668.557987:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 399 00000100:00100000:2.0:1789302668.557997:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350237696:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302668.558096:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350237696:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 100us (1663us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.558105:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 399 00000100:00100000:2.0:1789302668.558596:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238080 00000100:00100000:2.0:1789302668.558603:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238080 00000100:00100000:2.0:1789302668.558848:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238336 00000100:00100000:2.0:1789302668.558862:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238336 00000100:00100000:2.0:1789302668.558886:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:2.0:1789302668.558896:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350238464:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302668.558953:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350238464:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 58us (2370us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.558960:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:2.0:1789302668.558986:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:2.0:1789302668.558993:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350238080:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302668.559036:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350238080:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (2191us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.559050:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:2.0:1789302668.559063:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:2.0:1789302668.559069:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350238336:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302668.559091:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350238336:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (2214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.559098:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:2.0:1789302668.559120:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238208 00000100:00100000:2.0:1789302668.559124:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238208 00000100:00100000:2.0:1789302668.559155:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350237824 00000100:00100000:2.0:1789302668.559158:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350237824 00000100:00100000:2.0:1789302668.559178:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 310 00000100:00100000:2.0:1789302668.559185:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2348:x1876219350238208:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302668.559215:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2348:x1876219350238208:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (2657us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.559221:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 310 00000100:00100000:3.0:1789302668.559729:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350237952:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 2697us (3233us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.559740:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00080000:3.0:1789302668.561062:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302668.561085:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302668.561088:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302668.563021:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00100000:3.0:1789302668.563052:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:2348:x1876219350237824:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302668.563156:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:2348:x1876219350237824:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 123us (6554us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.563164:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 311 00000020:00000080:1.0:1789302668.732384:0:17441:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007fff8687b650 20000000:00000080:1.0:1789302668.732411:0:17441:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff960f3f7ee800 uarg=0000000000000000 00000040:00100000:1.0:1789302668.733677:0:17441:0:(llog.c:1072:llog_reverse_process()) index 44 : idx 1 20000000:01000000:1.0:1789302668.733691:0:17441:0:(mgs_llog.c:963:mgs_search_pool_cb()) Matched pool rec 43 cmd:0xce020 lustre.test_11c 00000040:00080000:1.0:1789302668.734355:0:17441:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].45, 224 off16008 20000000:01000000:1.0:1789302668.734387:0:17441:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce021 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1789302668.734432:0:17441:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].46, 136 off16144 00000040:00080000:1.0:1789302668.734954:0:17441:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].48, 224 off16608 00000040:00080000:1.0:1789302668.735401:0:17441:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].40, 224 off15280 20000000:01000000:1.0:1789302668.735415:0:17441:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce021 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1789302668.735471:0:17441:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].41, 144 off15424 00000040:00080000:1.0:1789302668.735621:0:17441:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].42, 224 off15648 10000000:01000000:1.0:1789302668.735638:0:17441:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:1.0:1789302668.735665:0:17441:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735690:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735699:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735706:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735715:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184c9fe expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735722:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca0c expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735729:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca1a expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735737:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca28 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735745:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca36 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735833:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca44 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735841:0:17441:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca52 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735850:0:17441:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735901:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca52 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.735918:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca52 expref: 18 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.735937:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371070720:0@lo:104: 00000100:00100000:1.0:1789302668.736106:0:17441:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.736261:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070720 00000100:00100000:2.0:1789302668.736272:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070720 00010000:00010000:1.0:1789302668.736353:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca44 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.736363:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca44 expref: 18 pid: 5836 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.736372:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371070848:0@lo:104: 00000100:00100000:1.0:1789302668.736400:0:17441:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302668.736423:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca36 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.736431:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca36 expref: 18 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.736449:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371070976:0@lo:104: 00000100:00100000:1.0:1789302668.736473:0:17441:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302668.736490:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca28 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.736498:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca28 expref: 18 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.736534:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef30700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071104:0@lo:104: 00000100:00100000:1.0:1789302668.736555:0:17441:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302668.736722:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070848 00000100:00100000:0.0:1789302668.736728:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070848 00000100:00100000:2.0:1789302668.736807:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371070976 00000100:00100000:2.0:1789302668.736811:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371070976 00010000:00010000:1.0:1789302668.736917:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca1a expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.736926:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca1a expref: 18 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.736935:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef32680 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071232:0@lo:104: 00000100:00100000:1.0:1789302668.736961:0:17441:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302668.736988:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca0c expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.736996:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca0c expref: 18 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.737004:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7e300 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071360:0@lo:104: 00000100:00100000:1.0:1789302668.737023:0:17441:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302668.737108:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184c9fe expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.737129:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184c9fe expref: 18 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.737138:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7dc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071488:0@lo:104: 00000100:00100000:1.0:1789302668.737166:0:17441:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302668.737188:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.737199:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 12 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.737210:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071616:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302668.737388:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.737397:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 12 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.737405:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ce00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071744:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302668.737437:0:17441:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.737445:0:17441:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 12 pid: 5836 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.737453:0:17441:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071872:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302668.737497:0:17441:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f087d7a80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302668.737625:0:5828:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1789302668.737639:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371071104 00000100:00100000:0.0:1789302668.737644:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371071104 00000100:00100000:0.0:1789302668.737659:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371071232 00000100:00100000:0.0:1789302668.737661:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371071232 00000100:00100000:0.0:1789302668.737669:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371071360 00000100:00100000:0.0:1789302668.737670:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371071360 00000100:00100000:0.0:1789302668.737678:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371071488 00000100:00100000:0.0:1789302668.737680:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371071488 00000100:00100000:0.0:1789302668.737694:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 35 00000100:00100000:0.0:1789302668.737703:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+9:17441:x1876219371070720:12345-0@lo:104: 00010000:00010000:0.0:1789302668.737720:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302668.737864:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1789302668.737871:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+9:17441:x1876219371070720:12345-0@lo:104: Request processed in 169us (1832us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302668.737879:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 35 00000100:00100000:0.0:1789302668.737901:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 36 00000100:00100000:0.0:1789302668.737906:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+8:17441:x1876219371070848:12345-0@lo:104: 00010000:00010000:0.0:1789302668.737924:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302668.737962:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:0.0:1789302668.737978:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+8:17441:x1876219371070848:12345-0@lo:104: Request processed in 72us (1578us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302668.737984:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 36 00000100:00100000:0.0:1789302668.738006:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:0.0:1789302668.738013:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+7:17441:x1876219371071104:12345-0@lo:104: 00010000:00010000:0.0:1789302668.738021:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302668.738124:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:0.0:1789302668.738130:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+7:17441:x1876219371071104:12345-0@lo:104: Request processed in 117us (1573us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302668.738136:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000100:00100000:0.0:1789302668.738151:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:0.0:1789302668.738157:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17441:x1876219371071232:12345-0@lo:104: 00010000:00010000:0.0:1789302668.738165:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302668.738194:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to priority blp list, 4 blwis in pool 00000100:00100000:0.0:1789302668.738200:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17441:x1876219371071232:12345-0@lo:104: Request processed in 43us (1238us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302668.738206:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000100:00100000:0.0:1789302668.738218:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:0.0:1789302668.738490:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17441:x1876219371071360:12345-0@lo:104: 00010000:00010000:0.0:1789302668.738563:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302668.738598:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to priority blp list, 5 blwis in pool 00000100:00100000:0.0:1789302668.738604:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17441:x1876219371071360:12345-0@lo:104: Request processed in 380us (1580us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302668.738610:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000100:00100000:0.0:1789302668.738625:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:0.0:1789302668.738632:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3b085850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17441:x1876219371071488:12345-0@lo:104: 00010000:00010000:0.0:1789302668.738639:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302668.738666:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to priority blp list, 6 blwis in pool 00000100:00100000:0.0:1789302668.738671:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3b085850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17441:x1876219371071488:12345-0@lo:104: Request processed in 41us (1504us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302668.738676:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00000100:00100000:0.0:1789302668.738689:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00000100:00100000:0.0:1789302668.738694:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17441:x1876219371070976:12345-0@lo:104: 00010000:00010000:0.0:1789302668.738702:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.739130:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371070720:0@lo:104: 00000100:00100000:1.0:1789302668.739143:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114ad80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371070848:0@lo:104: 00010000:00010000:1.0:1789302668.739181:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:1.0:1789302668.739190:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739198:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302668.739204:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739214:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302668.739330:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302668.739338:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302668.739392:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739400:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302668.739427:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739435:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e03c2c400/0x64fe43095184ca52 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca59 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739452:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1789302668.739456:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739478:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fe00 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302668.739481:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739489:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302668.739495:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302668.739525:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302668.739542:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739548:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302668.739555:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739561:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e0221fe00/0x64fe43095184ca44 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca4b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739570:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1789302668.739573:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739579:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302668.739581:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739597:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302668.739603:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302668.739608:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302668.739623:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739629:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302668.739644:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739651:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184ca28 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca2f expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739659:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1789302668.739663:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739668:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221f200 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302668.739671:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739677:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302668.739683:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302668.739695:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302668.739710:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739715:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302668.739722:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739729:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e0221f200/0x64fe43095184ca1a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca21 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739738:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1789302668.739742:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739747:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302668.739750:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739770:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302668.739776:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302668.739782:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302668.739794:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739800:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302668.739812:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739818:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e01bf2800/0x64fe43095184ca0c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca13 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739827:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302668.739830:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739836:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2de00 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:1.0:1789302668.739839:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739845:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:1.0:1789302668.739851:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302668.739857:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:1.0:1789302668.739871:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739876:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1789302668.739880:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739886:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e03c2de00/0x64fe43095184c9fe lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca05 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.739895:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302668.739901:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302668.739903:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302668.739923:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371072256:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302668.739993:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302668.740166:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302668.740173:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371072384:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302668.740201:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302668.740317:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072256 00000100:00100000:1.0:1789302668.740326:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072256 00010000:00010000:1.0:1789302668.740361:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca28 expref: 19 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.740387:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072384 00000100:00100000:1.0:1789302668.740390:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072384 00010000:00010000:1.0:1789302668.740402:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca1a expref: 20 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.740423:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1789302668.740431:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+20:3300:x1876219371072256:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789302668.740445:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302668.740451:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca28 expref: 20 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.740461:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca28 expref: 20 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.740481:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302668.740547:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+20:3300:x1876219371072256:12345-0@lo:103:ldlm_bl.0 Request processed in 117us (554us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302668.740555:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1789302668.740580:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789302668.740586:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371072384:12345-0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789302668.740595:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302668.740599:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca1a expref: 19 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.740608:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca1a expref: 19 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.740618:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302668.740641:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371072384:12345-0@lo:103:ldlm_bl.0 Request processed in 55us (439us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302668.740647:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00010000:00010000:1.0:1789302668.740713:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302668.740741:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371072384:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302668.741030:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371072256:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.741260:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072000:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.741304:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.741338:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072512:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.741360:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.741391:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789302668.741398:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072128:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.741419:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.741434:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072640:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.741452:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.741479:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072000 00000100:00100000:2.0:1789302668.741486:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072000 00010000:00010000:2.0:1789302668.741532:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca52 expref: 19 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302668.741555:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072512 00000100:00100000:2.0:1789302668.741558:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072512 00010000:00010000:2.0:1789302668.741571:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca0c expref: 20 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302668.741582:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072128 00000100:00100000:2.0:1789302668.741584:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072128 00010000:00010000:2.0:1789302668.741593:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca44 expref: 21 pid: 5836 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302668.741602:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072640 00000100:00100000:2.0:1789302668.741604:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072640 00010000:00010000:2.0:1789302668.741614:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184c9fe expref: 22 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302668.741626:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1789302668.741634:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+22:3302:x1876219371072000:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302668.741643:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302668.741648:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca52 expref: 22 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741673:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca52 expref: 22 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741686:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb800/0x64fe43095184ca59 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca52 expref: 22 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741700:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302668.741733:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+21:3302:x1876219371072000:12345-0@lo:103:ldlm_bl.0 Request processed in 100us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.741740:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1789302668.741770:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1789302668.741777:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+20:3302:x1876219371072512:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302668.741786:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302668.741790:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca0c expref: 20 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741799:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca0c expref: 20 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741809:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302668.741830:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+20:3302:x1876219371072512:12345-0@lo:103:ldlm_bl.0 Request processed in 54us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.741835:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:2.0:1789302668.741849:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789302668.741854:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371072128:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302668.741862:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302668.741866:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca44 expref: 19 pid: 5836 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741881:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca44 expref: 19 pid: 5836 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741892:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f3fcbf800/0x64fe43095184ca4b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca44 expref: 19 pid: 5836 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741903:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302668.741925:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:3302:x1876219371072128:12345-0@lo:103:ldlm_bl.0 Request processed in 71us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.741932:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:2.0:1789302668.741947:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789302668.741952:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3302:x1876219371072640:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302668.741960:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302668.741972:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184c9fe expref: 17 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741982:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184c9fe expref: 17 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.741992:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302668.742012:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3302:x1876219371072640:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (559us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.742017:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789302668.742067:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072128:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.742076:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072640:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.742083:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072000:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.742104:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371072512:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302668.742702:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1789302668.742713:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17441:x1876219371070976:12345-0@lo:104: Request processed in 4016us (6236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302668.742724:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00010000:00010000:2.0:1789302668.742908:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302668.742917:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.742925:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea600 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302668.742930:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.742938:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789302668.742946:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302668.742952:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789302668.743008:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.743015:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789302668.743102:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.743110:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184ca36 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca3d expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.743130:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302668.743132:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302668.743134:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302668.743141:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302668.743167:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371072768:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302668.743214:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302668.743341:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072768 00000100:00100000:2.0:1789302668.743349:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072768 00010000:00010000:2.0:1789302668.743377:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca36 expref: 17 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302668.743402:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789302668.743409:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3301:x1876219371072768:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302668.743423:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302668.743428:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 5/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca36 expref: 17 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.743439:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca36 expref: 17 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.743452:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302668.743481:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3301:x1876219371072768:12345-0@lo:103:ldlm_bl.0 Request processed in 73us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.743488:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789302668.743548:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371072768:0@lo:103:ldlm_bl.0 00010000:00010000:1.0:1789302668.747188:0:17441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f3fcbe800/0x64fe43095184ca3d lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca36 expref: 16 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.747216:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371070976:0@lo:104: 00010000:00010000:1.0:1789302668.747245:0:17441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb000/0x64fe43095184ca2f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca28 expref: 15 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.747254:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef30700 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071104:0@lo:104: 00010000:00010000:1.0:1789302668.747265:0:17441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca21 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca1a expref: 14 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.747273:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef32680 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071232:0@lo:104: 00010000:00010000:1.0:1789302668.747305:0:17441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e01bf2a00/0x64fe43095184ca13 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca0c expref: 13 pid: 5837 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.747315:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7e300 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071360:0@lo:104: 00010000:00010000:1.0:1789302668.747326:0:17441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb200/0x64fe43095184ca05 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184c9fe expref: 12 pid: 5838 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302668.747335:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7dc00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071488:0@lo:104: 00000100:00100000:1.0:1789302668.747348:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071616:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302668.747361:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ce00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071744:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302668.747371:0:17441:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17441:1876219371071872:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302668.747430:0:17441:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302668.750479:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302668.750616:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302668.750927:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238592 00000100:00100000:3.0:1789302668.750940:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238592 00010000:00010000:3.0:1789302668.751000:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 13 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302668.751050:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238848 00000100:00100000:3.0:1789302668.751054:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238848 00010000:00010000:3.0:1789302668.751069:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 14 pid: 5836 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302668.751096:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 219 00000100:00100000:3.0:1789302668.751106:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:2347:x1876219350238592:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302668.751123:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302668.751130:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 14 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302668.751143:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 14 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302668.751164:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eba00/0x64fe43095184c9f7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd0 expref: 14 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302668.751208:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302668.751285:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350238592:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 179us (819us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.751293:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 219 00000100:00100000:3.0:1789302668.751312:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 220 00000100:00100000:3.0:1789302668.751319:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2347:x1876219350238848:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302668.751327:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302668.751332:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 12 pid: 5836 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302668.751341:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 12 pid: 5836 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302668.751352:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e01bf3000/0x64fe43095184c9e9 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc2 expref: 12 pid: 5836 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302668.751362:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302668.751380:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:2347:x1876219350238848:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 62us (764us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302668.751387:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 220 00000100:00100000:0.0:1789302668.756764:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302668.756915:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238720 00000100:00100000:2.0:1789302668.756929:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238720 00010000:00010000:2.0:1789302668.756987:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 11 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302668.757035:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 221 00000100:00100000:2.0:1789302668.757071:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:2347:x1876219350238720:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302668.757091:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302668.757099:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 11 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757115:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 11 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757136:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e01bf3c00/0x64fe43095184c9f0 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffc9 expref: 11 pid: 5838 timeout: 605 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757160:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757171:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757190:0:5831:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757198:0:5831:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757206:0:5831:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757214:0:5831:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757221:0:5831:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302668.757234:0:5831:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757250:0:5831:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757272:0:5831:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302668.757286:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302668.757368:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+10:2347:x1876219350238720:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 298us (627us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302668.757377:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 221 00010000:00010000:1.0:1789302668.757448:0:17441:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.757459:0:17441:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302668.757466:0:17441:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb400/0x64fe43095184ca60 lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17441 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302668.815108:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302668.815206:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371072896:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302668.815364:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302668.818726:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371072896 00000100:00100000:0.0:1789302668.818839:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371072896 00000100:00100000:1.0:1789302668.819328:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 132 00000100:00100000:1.0:1789302668.819344:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371072896:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302668.819403:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302668.819653:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371072896:12345-0@lo:13:osp-pre-1-0.0 Request processed in 312us (4287us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302668.819679:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 132 00000100:00100000:3.0:1789302668.821513:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371072896:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302668.940768:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302669.261447:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302669.261541:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302669.265581:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 17 count 1 00000040:00080000:0.0:1789302669.265604:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302669.325085:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371073024:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302669.325201:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302669.325374:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073024 00000100:00100000:0.0:1789302669.325384:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073024 00000100:00100000:0.0:1789302669.325444:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 133 00000100:00100000:0.0:1789302669.325465:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371073024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302669.325542:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302669.325598:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371073024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 144us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302669.325605:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 133 00000100:00100000:2.0:1789302669.325957:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371073024:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302669.964835:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1789302670.732836:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1789302670.732849:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f120ab000 from 45 10000000:01000000:0.0:1789302670.732860:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1789302670.732959:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.732973:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.732986:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302670.733028:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073152:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.733128:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.733218:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599480 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.733344:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073152 00000100:00100000:1.0:1789302670.733364:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073152 00000100:00100000:1.0:1789302670.733435:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:1.0:1789302670.733446:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+12:5841:x1876219371073152:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302670.733465:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302670.733483:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca67 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.733552:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca67 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.733571:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca67 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.733584:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb400, rc 0) 00000100:00100000:1.0:1789302670.733629:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371073152:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 185us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.733640:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:0.0:1789302670.734345:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073152:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302670.734487:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.734498:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302670.734613:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073280:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.734659:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.735122:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599480 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.744363:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073280 00000100:00100000:3.0:1789302670.744415:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073280 00000100:00100000:3.0:1789302670.744527:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:3.0:1789302670.744540:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371073280:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.745430:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371073280:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 890us (10767us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.745442:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789302670.746396:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073280:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.746472:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073408:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.746582:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.746641:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302670.746762:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073408 00000100:00100000:2.0:1789302670.746771:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073408 00000100:00100000:2.0:1789302670.746827:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:2.0:1789302670.746838:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371073408:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.747089:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371073408:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 251us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302670.747097:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789302670.748770:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073408:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.750163:0:17444:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17444:1876219371073536:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302670.750275:0:17444:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.750550:0:17444:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f334119c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.750774:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073536 00000100:00100000:3.0:1789302670.750785:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073536 00000100:00100000:3.0:1789302670.750850:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789302670.750862:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17444:x1876219371073536:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.751669:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17444:x1876219371073536:12345-0@lo:502:llog_process_th.0 Request processed in 455us (1040us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.751682:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:1.0:1789302670.757224:0:17444:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17444:1876219371073536:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302670.757354:0:17444:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17444:1876219371073664:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302670.757458:0:17444:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302670.757563:0:17444:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0c757cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302670.757937:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073664 00000100:00100000:2.0:1789302670.757963:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073664 00000100:00100000:2.0:1789302670.758059:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1789302670.758071:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17444:x1876219371073664:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302670.758357:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17444:x1876219371073664:12345-0@lo:502:llog_process_th.0 Request processed in 287us (898us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302670.758365:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:1.0:1789302670.758671:0:17444:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17444:1876219371073664:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302670.758724:0:17444:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17444:1876219371073792:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302670.758771:0:17444:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302670.758809:0:17444:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08772780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.759674:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073792 00000100:00100000:3.0:1789302670.759685:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073792 00000100:00100000:3.0:1789302670.759754:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1789302670.759765:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17444:x1876219371073792:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.760210:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17444:x1876219371073792:12345-0@lo:502:llog_process_th.0 Request processed in 445us (1436us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.760219:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:0.0:1789302670.760348:0:17444:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17444:1876219371073792:0@lo:502:llog_process_th.0 00000040:00080000:0.0:1789302670.760389:0:17444:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000020:01000000:3.0:1789302670.760628:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 45-48 (rc=0) 10000000:01000000:3.0:1789302670.760656:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1789302670.760669:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.760681:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302670.760708:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:3.0:1789302670.760711:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:3.0:1789302670.760714:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:3.0:1789302670.760756:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.760765:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.760776:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302670.760798:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073920:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.760847:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302670.760891:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.763273:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371073920 00000100:00100000:1.0:1789302670.763295:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371073920 00000100:00100000:1.0:1789302670.763372:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1789302670.763384:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371073920:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302670.763407:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302670.763435:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca75 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.763472:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca75 expref: 14 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.763484:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca75 expref: 14 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.763495:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb200, rc 0) 00000100:00100000:1.0:1789302670.763627:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371073920:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 245us (2780us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.763634:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789302670.764252:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371073920:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302670.764281:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302670.764294:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302670.764406:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074048:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.764461:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.764537:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33411300 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.766635:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074048 00000100:00100000:1.0:1789302670.766658:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074048 00000100:00100000:1.0:1789302670.766738:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:1.0:1789302670.766751:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371074048:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.767813:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371074048:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 1059us (3345us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.767826:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789302670.768754:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074048:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.768833:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074176:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.768916:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.768985:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33411300 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.769286:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074176 00000100:00100000:3.0:1789302670.769296:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074176 00000100:00100000:3.0:1789302670.769358:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:3.0:1789302670.769368:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371074176:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.769670:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371074176:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 302us (753us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.769679:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:2.0:1789302670.771259:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074176:0@lo:503:ll_cfg_requeue.0 00000040:00080000:2.0:1789302670.774597:0:17445:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:3.0:1789302670.774918:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:3.0:1789302670.774959:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:3.0:1789302670.774976:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.774992:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302670.775030:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1789302670.775034:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f1d5a8000 from 45 10000000:01000000:3.0:1789302670.775037:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1789302670.775083:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.775093:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.775107:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302670.775141:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074304:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.775234:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302670.775289:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.775655:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074304 00000100:00100000:1.0:1789302670.775673:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074304 00000100:00100000:1.0:1789302670.775732:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1789302670.775743:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371074304:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302670.775760:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302670.775776:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca83 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.775807:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca83 expref: 15 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.775820:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca83 expref: 15 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.775829:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebe00, rc 0) 00000100:00100000:1.0:1789302670.775889:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371074304:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 148us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.775896:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1789302670.776744:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074304:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302670.776773:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302670.777153:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302670.777256:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074432:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.777316:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.777443:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f334119c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302670.783830:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074432 00000100:00100000:0.0:1789302670.783854:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074432 00000100:00100000:0.0:1789302670.783933:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:0.0:1789302670.783963:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371074432:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.784426:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371074432:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 482us (7109us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302670.784435:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:2.0:1789302670.784726:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074432:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.784796:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074560:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.784855:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.784890:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f334119c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.785046:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074560 00000100:00100000:3.0:1789302670.785053:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074560 00000100:00100000:3.0:1789302670.785095:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:3.0:1789302670.785104:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371074560:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.785309:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371074560:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 205us (454us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.785316:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:2.0:1789302670.785656:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371074560:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.787077:0:17446:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17446:1876219371074688:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302670.787179:0:17446:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.787229:0:17446:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33411240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.787341:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074688 00000100:00100000:3.0:1789302670.787360:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074688 00000100:00100000:3.0:1789302670.787413:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:3.0:1789302670.787423:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17446:x1876219371074688:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.787772:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17446:x1876219371074688:12345-0@lo:502:llog_process_th.0 Request processed in 349us (591us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.787790:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1789302670.792763:0:17446:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17446:1876219371074688:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302670.792911:0:17446:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17446:1876219371074816:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302670.793038:0:17446:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.793097:0:17446:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.793178:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074816 00000100:00100000:1.0:1789302670.793187:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074816 00000100:00100000:1.0:1789302670.793256:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1789302670.793268:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17446:x1876219371074816:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302670.793597:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17446:x1876219371074816:12345-0@lo:502:llog_process_th.0 Request processed in 329us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.793605:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1789302670.793930:0:17446:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17446:1876219371074816:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302670.793999:0:17446:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17446:1876219371074944:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302670.794051:0:17446:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.794089:0:17446:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.794140:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371074944 00000100:00100000:3.0:1789302670.794146:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371074944 00000100:00100000:3.0:1789302670.794182:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:3.0:1789302670.794190:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17446:x1876219371074944:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.794363:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17446:x1876219371074944:12345-0@lo:502:llog_process_th.0 Request processed in 173us (311us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.794370:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1789302670.794942:0:17446:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17446:1876219371074944:0@lo:502:llog_process_th.0 00000040:00080000:0.0:1789302670.795001:0:17446:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000020:01000000:1.0:1789302670.795312:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 45-48 (rc=0) 10000000:01000000:1.0:1789302670.795340:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1789302670.795362:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.795393:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302670.795406:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:1.0:1789302670.795409:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:1.0:1789302670.795413:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:1.0:1789302670.795459:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.795469:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.795480:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302670.795533:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075072:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.795590:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302670.795628:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.796077:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075072 00000100:00100000:3.0:1789302670.796087:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075072 00000100:00100000:3.0:1789302670.796144:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:3.0:1789302670.796156:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371075072:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302670.796174:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302670.796193:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184ca91 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.796225:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184ca91 expref: 16 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.796239:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca91 expref: 16 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.796248:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2de00, rc 0) 00000100:00100000:3.0:1789302670.796297:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371075072:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 144us (707us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.796304:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:0.0:1789302670.796559:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302670.796646:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350238976 00000100:00100000:1.0:1789302670.796652:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350238976 00000100:00100000:1.0:1789302670.796680:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 416 00000100:00100000:1.0:1789302670.796687:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+10:5588:x1876219350238976:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302670.796694:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302670.796705:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffd7 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.796722:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 11 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.796751:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 11 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.796759:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:1.0:1789302670.796806:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350238976:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 119us (249us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.796813:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 416 00000100:00100000:1.0:1789302670.796858:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075072:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302670.796878:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.796889:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302670.796976:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075200:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.797019:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302670.797149:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.797163:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075200 00000100:00100000:1.0:1789302670.797167:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075200 00000100:00100000:1.0:1789302670.797185:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:1.0:1789302670.797192:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371075200:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.797546:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371075200:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 354us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.797554:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:1.0:1789302670.797586:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075200:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.797629:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075328:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.797671:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302670.797685:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.797696:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075328 00000100:00100000:1.0:1789302670.797700:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075328 00000100:00100000:1.0:1789302670.797717:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:1.0:1789302670.797722:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371075328:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.797880:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371075328:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 157us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.797889:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:1.0:1789302670.797922:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075328:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.800757:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302670.802773:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350239104 00000100:00100000:3.0:1789302670.802810:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350239104 00000100:00100000:3.0:1789302670.802893:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 419 00000100:00100000:3.0:1789302670.802906:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350239104:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.803381:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350239104:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 475us (2635us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.803390:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 419 00000040:00080000:1.0:1789302670.812779:0:17447:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000020:01000000:0.0:1789302670.813144:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:0.0:1789302670.813169:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:0.0:1789302670.813183:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.813199:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302670.813213:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1789302670.813216:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f0511c000 from 45 10000000:01000000:0.0:1789302670.813220:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1789302670.813273:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.813284:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.813326:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302670.813362:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075456:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.813463:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.813545:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.814648:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075456 00000100:00100000:3.0:1789302670.814663:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075456 00000100:00100000:3.0:1789302670.814732:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:3.0:1789302670.814744:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371075456:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302670.814767:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302670.814791:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184caa6 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.814830:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184caa6 expref: 17 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.814843:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184caa6 expref: 17 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.814853:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2c400, rc 0) 00000100:00100000:3.0:1789302670.814937:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371075456:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 195us (1474us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.814963:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:0.0:1789302670.816817:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302670.816895:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075456:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302670.816932:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.816966:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302670.817088:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075584:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.817143:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.817197:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.817381:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350239232 00000100:00100000:1.0:1789302670.817392:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350239232 00000100:00100000:1.0:1789302670.817456:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075584 00000100:00100000:1.0:1789302670.817460:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075584 00000100:00100000:1.0:1789302670.817479:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 421 00000100:00100000:1.0:1789302670.817489:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350239232:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.817818:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350239232:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 329us (1013us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.817827:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 421 00000100:00100000:1.0:1789302670.817857:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1789302670.817863:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371075584:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.818178:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371075584:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 315us (1035us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.818197:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:0.0:1789302670.819793:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075584:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.819872:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075712:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.820283:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.820337:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.820520:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075712 00000100:00100000:1.0:1789302670.820530:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075712 00000100:00100000:1.0:1789302670.820588:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:1.0:1789302670.820599:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371075712:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.821162:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371075712:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 559us (875us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.821173:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:3.0:1789302670.830189:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371075712:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.830667:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302670.830754:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350239360 00000100:00100000:2.0:1789302670.830764:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350239360 00000100:00100000:2.0:1789302670.830834:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 424 00000100:00100000:2.0:1789302670.830846:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22474:x1876219350239360:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302670.831216:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22474:x1876219350239360:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 371us (553us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302670.831225:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 424 00000100:00100000:2.0:1789302670.832672:0:17448:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17448:1876219371075840:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302670.832764:0:17448:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.832824:0:17448:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.833608:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075840 00000100:00100000:1.0:1789302670.833622:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075840 00000100:00100000:1.0:1789302670.833695:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:1.0:1789302670.833708:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17448:x1876219371075840:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302670.834100:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17448:x1876219371075840:12345-0@lo:502:llog_process_th.0 Request processed in 394us (1334us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.834109:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789302670.834631:0:17448:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17448:1876219371075840:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.834717:0:17448:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17448:1876219371075968:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.834778:0:17448:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302670.834821:0:17448:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43780 going to sleep for 11 seconds 00000100:00100000:0.0:1789302670.834875:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371075968 00000100:00100000:0.0:1789302670.834883:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371075968 00000100:00100000:0.0:1789302670.834929:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:0.0:1789302670.834962:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17448:x1876219371075968:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302670.835163:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17448:x1876219371075968:12345-0@lo:502:llog_process_th.0 Request processed in 219us (384us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302670.835170:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:2.0:1789302670.835346:0:17448:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17448:1876219371075968:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302670.835386:0:17448:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17448:1876219371076096:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302670.835425:0:17448:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.835452:0:17448:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05480 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.835673:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076096 00000100:00100000:3.0:1789302670.835678:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076096 00000100:00100000:3.0:1789302670.835713:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1789302670.835721:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17448:x1876219371076096:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.835865:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17448:x1876219371076096:12345-0@lo:502:llog_process_th.0 Request processed in 144us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.835872:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:1.0:1789302670.836152:0:17448:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17448:1876219371076096:0@lo:502:llog_process_th.0 00000040:00080000:1.0:1789302670.836183:0:17448:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000020:01000000:3.0:1789302670.837368:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 45-48 (rc=0) 10000000:01000000:3.0:1789302670.837407:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1789302670.837420:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.837435:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302670.837450:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:3.0:1789302670.837453:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 40 10000000:01000000:3.0:1789302670.837456:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:3.0:1789302670.837549:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.837559:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.837571:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302670.837600:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076224:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.837706:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302670.837763:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.838260:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076224 00000100:00100000:1.0:1789302670.838271:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076224 00000100:00100000:1.0:1789302670.838369:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1789302670.838383:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371076224:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302670.838405:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302670.838428:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cab4 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.838461:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cab4 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.838487:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cab4 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.838498:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:1.0:1789302670.838590:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371076224:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 210us (883us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.838598:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:3.0:1789302670.838703:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076224:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302670.838725:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.838736:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302670.838825:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076352:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302670.838860:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.855881:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076352 00000100:00100000:2.0:1789302670.855902:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076352 00000100:00100000:2.0:1789302670.856590:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43780 going to sleep for 11 seconds 00000100:00100000:0.0:1789302670.857750:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:0.0:1789302670.857767:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371076352:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.859710:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371076352:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 1937us (20838us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302670.859725:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:0.0:1789302670.860252:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302670.860281:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350239488 00000100:00100000:0.0:1789302670.860292:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350239488 00000100:00100000:0.0:1789302670.860396:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 430 00000100:00100000:0.0:1789302670.860407:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22474:x1876219350239488:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:0.0:1789302670.860724:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22474:x1876219350239488:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 318us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302670.860732:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 430 00000100:00100000:2.0:1789302670.869895:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076352:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.870067:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076480:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.870194:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.870256:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302670.870345:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076480 00000100:00100000:0.0:1789302670.870356:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076480 00000100:00100000:0.0:1789302670.870420:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789302670.870432:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371076480:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.870733:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371076480:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 302us (538us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302670.870742:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:2.0:1789302670.873004:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076480:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.874454:0:17449:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17449:1876219371076608:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302670.874607:0:17449:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302670.874671:0:17449:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05540 going to sleep for 11 seconds 00000100:00100000:0.0:1789302670.874816:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076608 00000100:00100000:0.0:1789302670.874834:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076608 00000100:00100000:0.0:1789302670.874896:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:0.0:1789302670.874906:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17449:x1876219371076608:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302670.875251:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17449:x1876219371076608:12345-0@lo:502:llog_process_th.0 Request processed in 346us (643us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302670.875261:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:2.0:1789302670.877730:0:17449:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17449:1876219371076608:0@lo:502:llog_process_th.0 00000020:01000000:2.0:1789302670.877806:0:17449:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:2.0:1789302670.877828:0:17449:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:2.0:1789302670.877831:0:17449:0:(obd_config.c:1563:class_process_config()) marker 31 (0x1) lustre-MDT0000-m add lustre.test_11c.lustre-OST0001_UUID 00000020:00000080:2.0:1789302670.877840:0:17449:0:(obd_config.c:1472:class_process_config()) processing cmd: ce021 00020000:01000000:2.0:1789302670.877862:0:17449:0:(lod_pool.c:594:lod_pool_add()) Added lustre-OST0001_UUID to test_11c as member 1 00000100:00100000:1.0:1789302670.880522:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302670.881751:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350239616 00000100:00100000:2.0:1789302670.881770:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350239616 00000100:00100000:2.0:1789302670.881849:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00100000:2.0:1789302670.881862:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22474:x1876219350239616:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302670.882971:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22474:x1876219350239616:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 1102us (2465us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302670.882985:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00100000:3.0:1789302670.883451:0:17449:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17449:1876219371076736:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.883584:0:17449:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302670.883637:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076736 00000100:00100000:3.0:1789302670.883646:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076736 00000100:00100000:3.0:1789302670.883692:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1789302670.883701:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17449:x1876219371076736:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302670.883961:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17449:x1876219371076736:12345-0@lo:502:llog_process_th.0 Request processed in 248us (365us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.883968:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000020:01000004:1.0:1789302670.884242:0:17450:0:(tgt_mount.c:153:server_get_mount()) get mount ffff960f0511c000 from lustre-MDT0000, refs=4 00000020:01000004:1.0:1789302670.884248:0:17450:0:(obd_mount.c:710:lustre_put_lsi()) put ffff960f0511c000 4 00000100:00100000:3.0:1789302670.885466:0:17449:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43600 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.885539:0:17449:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cb480 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17449:1876219371076736:0@lo:502:llog_process_th.0 00000020:01000000:3.0:1789302670.885601:0:17449:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:3.0:1789302670.885613:0:17449:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:3.0:1789302670.885616:0:17449:0:(obd_config.c:1563:class_process_config()) marker 31 (0x2) lustre-MDT0000-m add lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:3.0:1789302670.885623:0:17449:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 43 count 43 00000020:01000000:0.0:1789302670.887874:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 40-42 (rc=0) 10000000:01000000:0.0:1789302670.887915:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:0.0:1789302670.887928:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.887959:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302670.887974:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1789302670.887977:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff960f050d19c0 from 1 10000000:01000000:0.0:1789302670.887982:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1789302670.888053:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.888063:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302670.888077:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302670.888109:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076864:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.888195:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302670.888252:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302670.888912:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076864 00000100:00100000:3.0:1789302670.888926:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076864 00000100:00100000:3.0:1789302670.889367:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:3.0:1789302670.889382:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371076864:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302670.889403:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302670.889428:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cac2 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.889464:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cac2 expref: 19 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.889476:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cac2 expref: 19 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302670.889487:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa200, rc 0) 00000100:00100000:3.0:1789302670.889595:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371076864:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 216us (1399us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302670.889602:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:2.0:1789302670.889701:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076864:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302670.889724:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302670.889735:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302670.889847:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076992:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302670.889886:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302670.903323:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371076992 00000100:00100000:1.0:1789302670.903361:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371076992 00000100:00100000:2.0:1789302670.903971:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302670.904275:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302670.904857:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350239744 00000100:00100000:1.0:1789302670.904867:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350239744 00000100:00100000:1.0:1789302670.904891:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1789302670.904905:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371076992:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.905333:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371076992:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 429us (15444us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302670.905343:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:2.0:1789302670.905601:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371076992:0@lo:501:ll_cfg_requeue.0 10000000:01000000:2.0:1789302670.905662:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:2.0:1789302670.905673:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302670.905685:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302670.906527:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00100000:1.0:1789302670.906545:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350239744:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302670.906565:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302670.906588:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffde expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.906619:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.906651:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302670.906662:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb600, rc 0) 00000100:00100000:1.0:1789302670.906735:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350239744:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 197us (2466us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.906742:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00100000:2.0:1789302670.911824:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302670.911957:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350239872 00000100:00100000:1.0:1789302670.911972:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350239872 00000100:00100000:1.0:1789302670.912042:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:1.0:1789302670.912055:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350239872:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.912461:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350239872:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 406us (650us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.912471:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:2.0:1789302670.932255:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302670.932370:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350240000 00000100:00100000:1.0:1789302670.932387:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350240000 00000100:00100000:1.0:1789302670.932460:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:1.0:1789302670.932474:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350240000:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302670.932837:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350240000:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 365us (596us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302670.932847:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:1.0:1789302670.950263:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302670.950465:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350240128 00000100:00100000:2.0:1789302670.950479:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350240128 00000100:00100000:2.0:1789302670.950577:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 440 00000100:00100000:2.0:1789302670.950612:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22475:x1876219350240128:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302670.951044:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22475:x1876219350240128:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 432us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302670.951053:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 440 00000100:00100000:2.0:1789302670.954778:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302670.954910:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350240256 00000100:00100000:0.0:1789302670.954925:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350240256 00000100:00100000:0.0:1789302670.955011:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:0.0:1789302670.955025:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22475:x1876219350240256:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:0.0:1789302670.955403:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22475:x1876219350240256:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 379us (640us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302670.955411:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:3.0:1789302670.960283:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302670.970174:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350240384 00000100:00100000:1.0:1789302670.970200:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350240384 00000100:00100000:2.0:1789302670.971390:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 442 00000100:00100000:2.0:1789302670.971411:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22475:x1876219350240384:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302670.971839:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22475:x1876219350240384:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 434us (11572us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302670.971848:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 442 00000100:00100000:1.0:1789302670.988786:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302670.988923:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302670.990626:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350240512 00000100:00100000:2.0:1789302670.990647:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350240512 00000100:00100000:2.0:1789302670.990742:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 443 00000100:00100000:2.0:1789302670.990754:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350240512:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302670.990779:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302670.990805:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffe5 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302670.990849:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302670.990863:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302670.990873:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe000, rc 0) 00000100:00100000:2.0:1789302670.990976:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350240512:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 224us (2205us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302670.991173:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 443 00000100:00100000:3.0:1789302670.995459:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302670.995605:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350240640 00000100:00100000:0.0:1789302670.995619:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350240640 00000100:00100000:0.0:1789302670.995686:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 444 00000100:00100000:0.0:1789302670.995698:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350240640:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302670.996165:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350240640:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 467us (721us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302670.996176:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 444 00000100:00100000:0.0:1789302671.934079:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302671.934177:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302671.934705:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350240768 00000100:00100000:0.0:1789302671.934720:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350240768 00000100:00100000:0.0:1789302671.934793:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350241024 00000100:00100000:0.0:1789302671.934797:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350241024 00000100:00100000:0.0:1789302671.934828:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 731 00000100:00100000:0.0:1789302671.934841:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2346:x1876219350240768:12345-192.168.203.11@tcp:41:mkdir.0 00000004:00000024:0.0:1789302671.934896:0:5845:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 2259812352 avail 2257715200 00000100:00100000:0.0:1789302671.934996:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2346:x1876219350240768:12345-192.168.203.11@tcp:41:mkdir.0 Request processed in 157us (929us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302671.935004:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 731 00000100:00100000:0.0:1789302671.935037:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 732 00000100:00100000:0.0:1789302671.935045:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:22485:x1876219350241024:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:0.0:1789302671.935094:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302671.935135:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184c6d9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739febf expref: 74 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.935157:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:0.0:1789302671.935163:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302671.935531:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.935549:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.935559:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.935609:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 30 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.935625:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 30 pid: 5844 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302671.935642:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371077120:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302671.935723:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b6c0 going to sleep for 11 seconds 00000100:00080000:0.0:1789302671.935749:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302671.941052:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302671.941199:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371077120:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302671.941237:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302671.941697:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350241152 00000100:00100000:1.0:1789302671.941707:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350241152 00010000:00010000:1.0:1789302671.941771:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 31 pid: 5844 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302671.941816:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 222 00000100:00100000:1.0:1789302671.941829:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2347:x1876219350241152:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302671.941847:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302671.941855:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 31 pid: 5844 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.941869:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 31 pid: 5844 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.941895:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184c718 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fed4 expref: 31 pid: 5844 timeout: 608 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.941914:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:1.0:1789302671.941920:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.941931:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.941939:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.941988:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302671.942059:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:2347:x1876219350241152:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 234us (1022us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302671.942068:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 222 00000100:00080000:0.0:1789302671.942150:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302671.942387:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.942396:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.942426:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.942437:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.942445:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302671.945891:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 16384 00010000:00010000:2.0:1789302671.945920:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.945932:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.945952:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.945960:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221f000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302671.945967:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.945977:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.946007:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:2.0:1789302671.946010:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302671.946017:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.946025:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f000/0x64fe43095184cade lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302671.946049:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000552f1f25 reply state 000000007243bba9 transno 4294967776 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302671.946059:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302671.946190:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:22485:x1876219350241024:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 11141us (12008us total) trans 4294967776 rc 0/0 00000100:00100000:2.0:1789302671.946201:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 732 00010000:00010000:3.0:1789302671.948484:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302671.948532:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302671.948559:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302671.948565:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302671.948570:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302671.948578:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302671.948594:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3c4dff78 (ffff960f03927100) 00010000:00010000:3.0:1789302671.948607:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302671.948613:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302671.948619:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cae5 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302671.963395:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302671.963475:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350241280 00000100:00100000:1.0:1789302671.963489:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350241280 00000100:00100000:1.0:1789302671.963586:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 733 00000100:00100000:1.0:1789302671.963598:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+74:22486:x1876219350241280:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302671.963629:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302671.963657:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea400/0x64fe43095184caec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739ffec expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.963741:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184caf3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.963757:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184caf3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.963771:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184caf3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302671.963817:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302671.963821:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302671.963842:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184caf3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.963872:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea400/0x64fe43095184caec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffec expref: 76 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.963892:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184caf3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffec expref: 75 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302671.963924:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24a00) 00010000:00010000:1.0:1789302671.963927:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302671.963932:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eba00, rc 0) 00000100:00100000:1.0:1789302671.964031:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:22486:x1876219350241280:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 434us (645us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302671.964040:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 733 00000100:00100000:0.0:1789302671.968126:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302671.968708:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350241408 00000100:00100000:2.0:1789302671.968718:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350241408 00000100:00100000:2.0:1789302671.968785:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 734 00000100:00100000:2.0:1789302671.968796:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:22486:x1876219350241408:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302671.968819:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302671.968841:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cafa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fff3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.968907:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.968922:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.968958:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.968975:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.968983:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.968990:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969092:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb0f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969100:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb0f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969107:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb0f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302671.969137:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 16384 00000004:00200000:2.0:1789302671.969140:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302671.969157:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb0f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969169:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969176:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969194:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969201:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302671.969205:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969214:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969226:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24a00) 00010000:00010000:2.0:1789302671.969229:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302671.969234:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969241:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb01 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969254:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969261:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969267:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969272:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302671.969275:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969292:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969301:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3c4dff78 (ffff960f03927f00) 00010000:00010000:2.0:1789302671.969304:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302671.969307:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969313:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfa00/0x64fe43095184cb08 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969347:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cafa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fff3 expref: 77 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969362:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb0f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fff3 expref: 76 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.969369:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff960f03927800) 00010000:00010000:2.0:1789302671.969371:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302671.969374:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf800, rc 0) 00000100:00100000:2.0:1789302671.969442:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:22486:x1876219350241408:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 648us (1326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302671.969450:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 734 00000100:00100000:3.0:1789302671.981726:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302671.982883:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350241664 00000100:00100000:0.0:1789302671.982900:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350241664 00000100:00100000:0.0:1789302671.982978:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 735 00000100:00100000:0.0:1789302671.982989:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:22486:x1876219350241664:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302671.983011:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302671.983032:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f600/0x64fe43095184cb16 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f739fffa expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302671.983118:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 16384 00010000:00010000:0.0:1789302671.983230:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f600/0x64fe43095184cb16 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fffa expref: 78 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.983244:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24a00) 00010000:00010000:0.0:1789302671.983247:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302671.983252:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f600/0x64fe43095184cb16 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fffa expref: 78 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.983263:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f600, rc 0) 00000100:00100000:0.0:1789302671.983322:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:22486:x1876219350241664:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 336us (1613us total) trans 4294967777 rc 301/301 00000100:00100000:0.0:1789302671.983329:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 735 00000100:00100000:0.0:1789302671.990191:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302671.990649:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350241920 00000100:00100000:2.0:1789302671.990660:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350241920 00000100:00100000:2.0:1789302671.990749:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 736 00000100:00100000:2.0:1789302671.990760:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:22486:x1876219350241920:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:2.0:1789302671.990821:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302671.990850:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb0f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fff3 expref: 77 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.990866:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff960f03927800) 00010000:00010000:2.0:1789302671.990876:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302671.990897:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.990909:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302671.990919:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302671.991133:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005378e7f0 reply state 0000000041a11f9b transno 4294967778 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302671.991144:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000004:00200000:2.0:1789302671.991153:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 16384 00000100:00100000:2.0:1789302671.991236:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:22486:x1876219350241920:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 477us (1051us total) trans 4294967778 rc 0/0 00000100:00100000:2.0:1789302671.991242:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 736 00000100:00100000:2.0:1789302671.997330:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302671.997362:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350242176 00000100:00100000:2.0:1789302671.997370:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350242176 00000100:00100000:2.0:1789302671.997418:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 151 00000100:00100000:2.0:1789302671.997428:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:22486:x1876219350242176:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:2.0:1789302671.997661:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:22486:x1876219350242176:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 233us (338us total) trans 4294967779 rc 0/0 00000100:00100000:2.0:1789302671.997676:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 151 00010000:00010000:0.0:1789302671.998273:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.998284:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.998300:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.998306:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302671.998313:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.998322:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.998339:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302671.998343:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302671.998348:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302671.998355:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf000/0x64fe43095184cb24 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.012613:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302672.405109:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302672.405182:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350242432 00000100:00100000:0.0:1789302672.405196:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350242432 00000100:00100000:0.0:1789302672.405270:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 737 00000100:00100000:0.0:1789302672.405282:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:22524:x1876219350242432:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:0.0:1789302672.405363:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302672.405400:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184caf3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffec expref: 78 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.405425:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24a00) 00010000:00010000:0.0:1789302672.405439:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302672.405978:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:22524:x1876219350242432:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 696us (882us total) trans 0 rc -17/-17 00000100:00100000:0.0:1789302672.405986:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 737 00000100:00100000:0.0:1789302672.413469:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302672.413814:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350242560 00000100:00100000:2.0:1789302672.413828:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350242560 00000100:00100000:2.0:1789302672.413953:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 738 00000100:00100000:2.0:1789302672.413966:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:22524:x1876219350242560:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:2.0:1789302672.413996:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302672.414038:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb2b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0001 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.414127:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb32 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.414144:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.414157:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302672.414206:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302672.414210:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302672.414232:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.414260:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cb2b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0001 expref: 79 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.414285:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb32 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0001 expref: 78 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.414315:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:2.0:1789302672.414318:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302672.414323:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbec00, rc 0) 00000100:00100000:2.0:1789302672.414404:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:22524:x1876219350242560:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 440us (953us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302672.414412:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 738 00000100:00100000:3.0:1789302672.418170:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302672.418558:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350242688 00000100:00100000:0.0:1789302672.418571:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350242688 00000100:00100000:0.0:1789302672.418644:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 739 00000100:00100000:0.0:1789302672.418658:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:22524:x1876219350242688:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:0.0:1789302672.418685:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302672.418712:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cb39 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0008 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.418801:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.418820:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.418834:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.418852:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.418859:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.418866:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.419384:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cb4e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.419413:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cb4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.419421:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cb4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302672.419470:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2d:0x0]: returning size 16384 00000004:00200000:0.0:1789302672.419476:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302672.421231:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cb4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421264:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421270:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421276:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421281:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302672.421285:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421290:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421308:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:0.0:1789302672.421310:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302672.421314:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421341:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e000/0x64fe43095184cb40 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421356:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421359:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421365:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421370:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221e400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302672.421373:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421379:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421387:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3c4dff78 (ffff960f03501c00) 00010000:00010000:0.0:1789302672.421389:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302672.421392:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421399:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb47 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3c4dff78 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421465:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cb39 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0008 expref: 80 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421482:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cb4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0008 expref: 79 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.421493:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff960f03501a00) 00010000:00010000:0.0:1789302672.421494:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302672.421497:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f200, rc 0) 00000100:00100000:0.0:1789302672.421636:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:22524:x1876219350242688:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 2979us (3478us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302672.421647:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 739 00000100:00100000:3.0:1789302672.474215:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302672.474402:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350242816 00000100:00100000:0.0:1789302672.474415:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350242816 00000100:00100000:0.0:1789302672.474487:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 740 00000100:00100000:0.0:1789302672.474498:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+30:22530:x1876219350242816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302672.474567:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302672.474605:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cb55 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a000f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.474704:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.474722:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.474736:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302672.474789:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302672.474792:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302672.474817:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.474867:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cb55 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.474893:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 31 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.474904:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:0.0:1789302672.474907:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302672.474913:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221e400, rc 0) 00000100:00100000:0.0:1789302672.475057:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:22530:x1876219350242816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 511us (809us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302672.475067:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 740 00000100:00100000:2.0:1789302672.482222:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302672.482399:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350243072 00000100:00100000:1.0:1789302672.482414:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350243072 00000100:00100000:1.0:1789302672.482556:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 741 00000100:00100000:1.0:1789302672.482569:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:22530:x1876219350243072:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302672.482609:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302672.482613:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789302672.482660:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cb4e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2d:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0008 expref: 79 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.482684:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff960f03501a00) 00010000:00010000:1.0:1789302672.482703:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302672.482725:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea400/0x64fe43095184cb63 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0016 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.482889:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.482905:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.482917:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.482967:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.482975:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.482982:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.484469:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.484489:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.484527:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302672.484541:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2f:0x0]: returning size 0 00010000:00010000:1.0:1789302672.484574:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.484582:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.484590:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.484964:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302672.484970:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.484999:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.485015:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302672.485081:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302672.485089:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.485096:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea600/0x64fe43095184cb6a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302672.485113:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000024c2c518 reply state 00000000c8b1114d transno 4294967780 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302672.485131:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302672.485194:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea400/0x64fe43095184cb63 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.485217:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 80 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.485225:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff960f10d24600) 00010000:00010000:1.0:1789302672.485227:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302672.485230:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea800, rc 0) 00000100:00100000:1.0:1789302672.486045:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:22530:x1876219350243072:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 3469us (3829us total) trans 4294967780 rc 0/0 00000100:00100000:1.0:1789302672.486058:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 741 00000100:00100000:2.0:1789302672.490606:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302672.490729:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.490746:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.490758:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.490765:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302672.490771:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.490781:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.490801:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0xecadbcfe (ffff960f10d24500) 00010000:00010000:2.0:1789302672.490805:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302672.490811:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.490845:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184cb71 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2d:0x0].0xecadbcfe bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302672.491918:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350243328 00000100:00100000:1.0:1789302672.491958:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350243328 00000100:00100000:1.0:1789302672.492539:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 742 00000100:00100000:1.0:1789302672.492555:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:22530:x1876219350243328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302672.492592:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302672.492632:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cb86 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a001d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.492778:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.492797:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.492808:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 4/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.492856:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.492879:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 81 pid: 6919 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302672.492895:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371077248:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302672.493094:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cfcc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302672.498076:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302672.498226:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350243456 00000100:00100000:0.0:1789302672.498242:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350243456 00010000:00010000:0.0:1789302672.498317:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 82 pid: 6919 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302672.498367:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 223 00000100:00100000:0.0:1789302672.498379:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:2348:x1876219350243456:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302672.498398:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302672.498406:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 82 pid: 6919 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.498421:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 82 pid: 6919 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.498476:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff960f10d24600) 00010000:00010000:0.0:1789302672.498482:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,1 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.498493:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.498801:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.498833:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302672.498908:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:2348:x1876219350243456:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 530us (854us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302672.498917:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 223 00010000:00010000:1.0:1789302672.500762:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184cb78 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0016 expref: 81 pid: 6919 timeout: 609 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302672.500790:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371077248:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302672.500844:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.500856:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.500861:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302672.500888:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x2f:0x0]: returning size 0 00010000:00010000:1.0:1789302672.502425:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502444:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502453:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502460:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eba00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302672.502465:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502475:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502494:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff960f10d24600) 00010000:00010000:1.0:1789302672.502498:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302672.502689:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 2/0,0 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502697:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eba00/0x64fe43095184cb8d lrc: 0/0,0 mode: --/EX res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502800:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cb86 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a001d expref: 81 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502811:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2d:0x0].0x0 (ffff960f10d24400) 00010000:00010000:1.0:1789302672.502814:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302672.502818:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cb86 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x2d:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a001d expref: 81 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.502832:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea200, rc 0) 00000100:00100000:1.0:1789302672.502986:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:22530:x1876219350243328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 10435us (12736us total) trans 4294967781 rc 301/301 00000100:00100000:1.0:1789302672.502997:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 742 00000100:00100000:2.0:1789302672.505858:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302672.505976:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350243712 00000100:00100000:3.0:1789302672.505989:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350243712 00000100:00100000:3.0:1789302672.506154:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 152 00000100:00100000:3.0:1789302672.506167:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:22530:x1876219350243712:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302672.506976:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:22530:x1876219350243712:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 808us (1128us total) trans 4294967782 rc 0/0 00000100:00100000:3.0:1789302672.506985:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 152 00000100:00100000:0.0:1789302672.509908:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302672.510008:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350243840 00000100:00100000:1.0:1789302672.510033:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350243840 00000100:00100000:1.0:1789302672.510114:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 743 00000100:00100000:1.0:1789302672.510127:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:22530:x1876219350243840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302672.510163:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302672.510198:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea400/0x64fe43095184cb9b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0024 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.510477:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea400/0x64fe43095184cb9b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0024 expref: 82 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.510498:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea400/0x64fe43095184cb9b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x2f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0024 expref: 82 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302672.510585:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:1.0:1789302672.510588:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302672.510712:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea400, rc 0) 00000100:00100000:1.0:1789302672.510845:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:22530:x1876219350243840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 721us (954us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302672.510854:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 743 00000100:00100000:0.0:1789302672.513728:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302672.513839:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350244096 00000100:00100000:2.0:1789302672.513852:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350244096 00000100:00100000:2.0:1789302672.513969:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 744 00000100:00100000:2.0:1789302672.513981:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:22530:x1876219350244096:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1789302672.514083:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514105:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514119:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514404:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514413:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514421:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514428:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302672.514434:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514461:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514477:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:2.0:1789302672.514480:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302672.514487:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302672.514495:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cba2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302672.514906:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:22530:x1876219350244096:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 924us (1189us total) trans 4294967783 rc 0/0 00000100:00100000:2.0:1789302672.514917:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 744 00000100:00100000:2.0:1789302672.518745:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302672.518880:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350244352 00000100:00100000:0.0:1789302672.518894:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350244352 00000100:00100000:0.0:1789302672.519038:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 745 00000100:00100000:0.0:1789302672.519051:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:22530:x1876219350244352:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302672.519433:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.519453:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302672.519466:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302672.519719:0:6919:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967784 00000004:00010000:0.0:1789302672.519767:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006b3e02e5 reply state 00000000afa3a013 transno 4294967785 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302672.519778:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302672.519945:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:22530:x1876219350244352:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 888us (1926us total) trans 4294967785 rc 0/0 00000100:00100000:0.0:1789302672.519953:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 745 00000020:00080000:0.0:1789302672.519959:0:6919:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:0.0:1789302672.526135:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302672.526296:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302672.526317:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302672.526328:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302672.526334:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221e600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302672.526340:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302672.526349:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302672.526372:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x2f:0x0].0x0 (ffff960f10d24b00) 00010000:00010000:3.0:1789302672.526375:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789302672.526382:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302672.526389:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184cba9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x2f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302672.526498:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350244608 00000100:00100000:2.0:1789302672.527390:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350244608 00000100:00100000:2.0:1789302672.527568:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 153 00000100:00100000:2.0:1789302672.527581:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:22530:x1876219350244608:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302672.528355:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:22530:x1876219350244608:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 770us (2228us total) trans 4294967786 rc 0/0 00000100:00100000:2.0:1789302672.528366:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 153 00010000:00010000:1.0:1789302673.036986:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302673.422459:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371077376:0@lo:400:kworker.0 00000100:00100000:0.0:1789302673.422715:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302673.422944:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371077504:0@lo:400:kworker.0 00000100:00100000:3.0:1789302673.422991:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302673.423029:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302673.423036:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371077632:0@lo:400:kworker.0 00000100:00100000:3.0:1789302673.423060:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302673.423103:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371077504 00000100:00100000:3.0:1789302673.423114:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371077504 00000100:00100000:3.0:1789302673.423178:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371077632 00000100:00100000:3.0:1789302673.423182:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371077632 00000100:00100000:3.0:1789302673.423218:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1789302673.423229:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371077504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302673.423306:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371077504:12345-0@lo:400:kworker.0 Request processed in 79us (313us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302673.423314:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:3.0:1789302673.423335:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1789302673.423342:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371077632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302673.423368:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371077632:12345-0@lo:400:kworker.0 Request processed in 27us (308us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302673.423374:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00100000:3.0:1789302673.423414:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371077632:0@lo:400:kworker.0 00000100:00100000:3.0:1789302673.423425:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371077504:0@lo:400:kworker.0 00000100:00100000:3.0:1789302673.425319:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371077760:0@lo:400:kworker.0 00000100:00100000:3.0:1789302673.425397:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302673.425458:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371077760 00000100:00100000:3.0:1789302673.425467:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371077760 00000100:00100000:3.0:1789302673.425557:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789302673.425568:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371077760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302673.426036:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371077376 00000100:00100000:0.0:1789302673.426043:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371077376 00000100:00100000:0.0:1789302673.426145:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:0.0:1789302673.426154:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371077376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302673.426222:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371077376:12345-0@lo:400:kworker.0 Request processed in 70us (3635us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302673.426229:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:3.0:1789302673.426393:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371077760:12345-0@lo:400:kworker.0 Request processed in 827us (995us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302673.426402:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789302673.426455:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371077760:0@lo:400:kworker.0 00000100:00080000:0.0:1789302673.426684:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302673.426691:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302673.426729:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371077376:0@lo:400:kworker.0 00000100:00100000:2.0:1789302673.427467:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302673.427482:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371077888:0@lo:400:kworker.0 00000100:00100000:2.0:1789302673.427555:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302673.427607:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371077888 00000100:00100000:2.0:1789302673.427613:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371077888 00000100:00100000:2.0:1789302673.427675:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1789302673.427683:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371077888:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302673.427727:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371077888:12345-0@lo:400:kworker.0 Request processed in 45us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302673.427734:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1789302673.427764:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371077888:0@lo:400:kworker.0 00000100:00100000:2.0:1789302673.427806:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371078016:0@lo:400:kworker.0 00000100:00100000:2.0:1789302673.427835:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302673.427868:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078016 00000100:00100000:2.0:1789302673.427872:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078016 00000100:00100000:2.0:1789302673.427900:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1789302673.427906:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371078016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302673.427952:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371078016:12345-0@lo:400:kworker.0 Request processed in 46us (115us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302673.427958:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1789302673.427986:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371078016:0@lo:400:kworker.0 00000100:00080000:2.0:1789302673.428361:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302673.428366:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302673.678753:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302673.678821:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302673.678855:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302673.678880:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302673.678902:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302673.678921:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302673.678956:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302673.678995:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350244992 00000100:00100000:3.0:1789302673.679015:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350244992 00000100:00100000:3.0:1789302673.679090:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350245504 00000100:00100000:3.0:1789302673.679094:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350245504 00000100:00100000:3.0:1789302673.679110:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350245120 00000100:00100000:3.0:1789302673.679113:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350245120 00000100:00100000:3.0:1789302673.679124:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350245376 00000100:00100000:3.0:1789302673.679127:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350245376 00000100:00100000:3.0:1789302673.679144:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 287 00000100:00100000:3.0:1789302673.679156:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350244992:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302673.679833:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350244992:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 676us (1007us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302673.679842:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 287 00000100:00100000:3.0:1789302673.679870:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:3.0:1789302673.679879:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350245504:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302673.679910:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350245504:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (1029us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302673.679917:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:3.0:1789302673.679929:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 289 00000100:00100000:3.0:1789302673.679946:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350245120:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302673.679972:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350245120:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (1050us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302673.679991:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 289 00000100:00100000:1.0:1789302673.680122:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350244736 00000100:00100000:1.0:1789302673.680130:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350244736 00000100:00100000:1.0:1789302673.680178:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:1.0:1789302673.680186:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350244736:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302673.680215:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350244736:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (1480us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302673.680222:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:1.0:1789302673.680291:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350245248 00000100:00100000:1.0:1789302673.680296:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350245248 00000100:00100000:1.0:1789302673.680328:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350244864 00000100:00100000:1.0:1789302673.680331:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350244864 00000100:00100000:1.0:1789302673.680355:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:1.0:1789302673.680362:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2346:x1876219350245248:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302673.680394:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2346:x1876219350245248:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (1537us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302673.680401:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:1.0:1789302673.680412:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 316 00000100:00100000:1.0:1789302673.680419:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:2346:x1876219350244864:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302673.680462:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:2346:x1876219350244864:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (1559us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302673.680470:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 316 00000100:00100000:3.0:1789302673.680753:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 290 00000100:00100000:3.0:1789302673.680763:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350245376:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302673.680798:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350245376:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (1841us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302673.680804:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 290 00000100:00080000:3.0:1789302673.680846:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302673.680852:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302673.680856:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302673.933845:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302673.933876:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371078144:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302673.933974:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302673.934060:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078144 00000100:00100000:1.0:1789302673.934069:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078144 00000100:00100000:1.0:1789302673.934122:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:1.0:1789302673.934131:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371078144:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302673.934179:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302673.934230:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371078144:12345-0@lo:13:osp-pre-1-0.0 Request processed in 99us (256us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302673.934237:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:1.0:1789302673.934292:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371078144:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302674.063872:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302674.444944:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302674.444986:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371078272:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302674.445084:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302674.445324:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078272 00000100:00100000:1.0:1789302674.445335:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078272 00000100:00100000:1.0:1789302674.445404:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 135 00000100:00100000:1.0:1789302674.445416:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371078272:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302674.445463:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302674.445584:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371078272:12345-0@lo:13:osp-pre-0-0.0 Request processed in 169us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302674.445592:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 135 00000100:00100000:0.0:1789302674.446135:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371078272:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302675.084672:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302676.109342:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789302677.120656:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967785 is committed 00000001:00080000:3.0:1789302677.120712:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967786 is committed 00010000:00010000:3.0:1789302677.134374:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302678.157235:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302678.546145:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371078784:0@lo:400:kworker.0 00000100:00100000:1.0:1789302678.546254:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302678.546330:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1789302678.546337:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371078400:0@lo:400:kworker.0 00000100:00100000:1.0:1789302678.546370:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302678.546393:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371078912:0@lo:400:kworker.0 00000100:00100000:1.0:1789302678.546419:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302678.546448:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371078528:0@lo:400:kworker.0 00000100:00100000:1.0:1789302678.546475:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302678.546492:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371079040:0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.546665:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078400 00000100:00100000:0.0:1789302678.546676:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078400 00000100:00100000:0.0:1789302678.546732:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1789302678.546743:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371078400:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.546808:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371078400:12345-0@lo:400:kworker.0 Request processed in 66us (436us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.546816:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1789302678.548601:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078912 00000100:00100000:0.0:1789302678.548620:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078912 00000100:00100000:0.0:1789302678.548693:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078528 00000100:00100000:0.0:1789302678.548697:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078528 00000100:00100000:0.0:1789302678.548727:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1789302678.548737:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371078912:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.548837:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371078912:12345-0@lo:400:kworker.0 Request processed in 101us (2415us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.548843:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00100000:0.0:1789302678.548865:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:0.0:1789302678.548870:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371078528:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.548910:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371078528:12345-0@lo:400:kworker.0 Request processed in 39us (2435us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.548916:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:0.0:1789302678.548937:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078784 00000100:00100000:0.0:1789302678.548941:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078784 00000100:00100000:0.0:1789302678.548985:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:0.0:1789302678.548992:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371078784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.549025:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371078784:12345-0@lo:400:kworker.0 Request processed in 34us (2772us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.549046:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:0.0:1789302678.549073:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371078656:0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.549115:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302678.549157:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371078656 00000100:00100000:0.0:1789302678.549160:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371078656 00000100:00100000:0.0:1789302678.549178:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 319 00000100:00100000:0.0:1789302678.549185:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371078656:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.549220:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371078656:12345-0@lo:400:kworker.0 Request processed in 35us (105us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.549226:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 319 00000100:00100000:0.0:1789302678.549268:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371078656:0@lo:400:kworker.0 00000100:00080000:3.0:1789302678.549700:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302678.549707:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302678.549811:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302678.551818:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371078528:0@lo:400:kworker.0 00000100:00100000:3.0:1789302678.558821:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371079040 00000100:00100000:3.0:1789302678.558844:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371079040 00000100:00100000:3.0:1789302678.558953:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:3.0:1789302678.558977:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371079040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302678.559106:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371079040:12345-0@lo:400:kworker.0 Request processed in 142us (9294us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302678.559114:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00080000:0.0:1789302678.559233:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302678.560652:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371078400:0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.560679:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371078912:0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.560692:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371078784:0@lo:400:kworker.0 00000100:00100000:1.0:1789302678.563639:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371079040:0@lo:400:kworker.0 00000100:00100000:0.0:1789302678.804387:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302678.804448:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302678.804483:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302678.804541:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302678.804691:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302678.804716:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302678.804745:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302678.804783:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350245888 00000100:00100000:0.0:1789302678.804797:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350245888 00000100:00100000:0.0:1789302678.805042:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246400 00000100:00100000:0.0:1789302678.805046:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246400 00000100:00100000:0.0:1789302678.805068:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246016 00000100:00100000:0.0:1789302678.805071:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246016 00000100:00100000:0.0:1789302678.805084:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246272 00000100:00100000:0.0:1789302678.805086:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246272 00000100:00100000:0.0:1789302678.805102:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:0.0:1789302678.805114:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f050a5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350245888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302678.805221:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f050a5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2345:x1876219350245888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 109us (772us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.805229:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:3.0:1789302678.806021:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350245632 00000100:00100000:3.0:1789302678.806034:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350245632 00000100:00100000:3.0:1789302678.806085:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 448 00000100:00100000:3.0:1789302678.806095:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350245632:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302678.806141:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350245632:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 49us (1774us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302678.806157:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 448 00000100:00100000:0.0:1789302678.806758:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 294 00000100:00100000:0.0:1789302678.806771:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350246400:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302678.806812:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350246400:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 45us (2270us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.806819:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 294 00000100:00100000:0.0:1789302678.806838:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 295 00000100:00100000:0.0:1789302678.806847:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350246016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302678.807659:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350246016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 806us (2934us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.807669:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 295 00000100:00100000:0.0:1789302678.807701:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 296 00000100:00100000:0.0:1789302678.808250:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350246272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302678.808319:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350246272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 609us (3571us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.808348:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 296 00000100:00100000:0.0:1789302678.808393:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246144 00000100:00100000:0.0:1789302678.808403:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246144 00000100:00100000:0.0:1789302678.808463:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350245760 00000100:00100000:0.0:1789302678.808467:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350245760 00000100:00100000:0.0:1789302678.808525:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:0.0:1789302678.808533:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2345:x1876219350246144:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302678.809059:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:2.0:1789302678.809072:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2345:x1876219350245760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302678.809118:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2345:x1876219350245760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 50us (4427us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302678.809125:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:0.0:1789302678.809661:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2345:x1876219350246144:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1126us (5174us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302678.809669:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00080000:0.0:1789302678.809715:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302678.809722:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302678.809725:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302679.053379:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371079168:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302679.053493:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302679.053764:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371079168 00000100:00100000:3.0:1789302679.053774:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371079168 00000100:00100000:3.0:1789302679.053841:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:3.0:1789302679.053853:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371079168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302679.053914:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302679.059147:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371079168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 5282us (5641us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302679.059167:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:0.0:1789302679.065872:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371079168:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302679.188278:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302679.500888:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302679.500907:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302679.501477:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302679.501489:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302679.580144:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302679.580173:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371079296:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302679.580270:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302679.580359:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371079296 00000100:00100000:3.0:1789302679.580370:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371079296 00000100:00100000:3.0:1789302679.580428:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 137 00000100:00100000:3.0:1789302679.580438:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3b2ba050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371079296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302679.580484:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302679.580565:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3b2ba050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371079296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 128us (294us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302679.580573:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 137 00000100:00100000:3.0:1789302679.580881:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371079296:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302680.204965:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302681.228770:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:0.0:1789302681.666322:0:17743:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007ffe62499890 20000000:00000080:0.0:1789302681.666411:0:17743:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff960f3fc48400 uarg=0000000000000000 00000040:00100000:0.0:1789302681.668482:0:17743:0:(llog.c:1072:llog_reverse_process()) index 48 : idx 48 00000040:00100000:0.0:1789302681.668541:0:17743:0:(llog.c:1072:llog_reverse_process()) index 46 : idx 1 20000000:01000000:0.0:1789302681.668546:0:17743:0:(mgs_llog.c:963:mgs_search_pool_cb()) Matched pool rec 46 cmd:0xce021 lustre.test_11c lustre-OST0001_UUID 20000000:01000000:0.0:1789302681.668559:0:17743:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-client/lustre-clilov/add lustre.test_11c.lustre-OST0001_UUID fl=4 20000000:01000000:2.0:1789302681.672084:0:17744:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 45 marker 30 1->4: lustre-clilov add lustre.test_11c.lustre-OST0001_UUID 20000000:01000000:2.0:1789302681.672244:0:17744:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 48 marker 30 2->4: lustre-clilov add lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:2.0:1789302681.672281:0:17744:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 49 count 48 00000040:00080000:1.0:1789302681.672847:0:17743:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].49, 224 off16832 20000000:01000000:1.0:1789302681.672864:0:17743:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce022 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1789302681.672934:0:17743:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].50, 136 off16968 00000040:00080000:1.0:1789302681.672975:0:17743:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].51, 224 off17192 20000000:01000000:1.0:1789302681.672994:0:17743:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-MDT0000/lustre-MDT0000-mdtlov/add lustre.test_11c.lustre-OST0001_UUID fl=4 20000000:01000000:0.0:1789302681.679229:0:17745:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 40 marker 31 1->4: lustre-MDT0000-mdtlov add lustre.test_11c.lustre-OST0001_UUID 20000000:01000000:0.0:1789302681.679364:0:17745:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 42 marker 31 2->4: lustre-MDT0000-mdtlov add lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:0.0:1789302681.679399:0:17745:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 43 count 43 00000040:00080000:1.0:1789302681.680010:0:17743:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].43, 224 off15872 20000000:01000000:1.0:1789302681.680027:0:17743:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce022 lustre test_11c lustre-OST0001_UUID (null) 00000040:00080000:1.0:1789302681.680078:0:17743:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].44, 144 off16016 00000040:00080000:1.0:1789302681.680116:0:17743:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].45, 224 off16240 10000000:01000000:1.0:1789302681.680131:0:17743:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:1.0:1789302681.680158:0:17743:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680176:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca67 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680184:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca75 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680191:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca83 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680198:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184ca91 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680208:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 12 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680214:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184caa6 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680220:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cab4 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680227:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cac2 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680233:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680241:0:17743:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680255:0:17743:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680291:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680306:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 12 pid: 5838 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680319:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079424:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302681.680475:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680485:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680493:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079552:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302681.680554:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cac2 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680562:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cac2 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680572:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079680:0@lo:104: 00000100:00100000:1.0:1789302681.680609:0:17743:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302681.680640:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cab4 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680659:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cab4 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680667:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079808:0@lo:104: 00000100:00100000:1.0:1789302681.680687:0:17743:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302681.680711:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184caa6 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680719:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184caa6 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680727:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079936:0@lo:104: 00000100:00100000:1.0:1789302681.680745:0:17743:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302681.680759:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 12 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680766:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 12 pid: 5837 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680773:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080064:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302681.680799:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca91 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680806:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca91 expref: 18 pid: 5836 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680813:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080192:0@lo:104: 00000100:00100000:1.0:1789302681.680831:0:17743:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302681.680854:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca83 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680861:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca83 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680868:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080320:0@lo:104: 00000100:00100000:1.0:1789302681.680902:0:17743:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302681.680917:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca75 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680925:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca75 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680932:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080448:0@lo:104: 00000100:00100000:1.0:1789302681.680949:0:17743:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:1.0:1789302681.680962:0:17743:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184ca67 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302681.680969:0:17743:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184ca67 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.680975:0:17743:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080576:0@lo:104: 00000100:00100000:1.0:1789302681.680996:0:17743:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302681.681006:0:17743:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58a3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302681.681209:0:5828:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:0.0:1789302681.681229:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371079680 00000100:00100000:0.0:1789302681.681236:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371079680 00000100:00100000:0.0:1789302681.681270:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371079808 00000100:00100000:0.0:1789302681.681273:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371079808 00000100:00100000:0.0:1789302681.681283:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371079936 00000100:00100000:0.0:1789302681.681285:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371079936 00000100:00100000:0.0:1789302681.681293:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371080192 00000100:00100000:0.0:1789302681.681295:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371080192 00000100:00100000:0.0:1789302681.681302:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371080320 00000100:00100000:0.0:1789302681.681304:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371080320 00000100:00100000:0.0:1789302681.681312:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371080448 00000100:00100000:0.0:1789302681.681314:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371080448 00000100:00100000:0.0:1789302681.681334:0:5828:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371080576 00000100:00100000:0.0:1789302681.681337:0:5828:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371080576 00000100:00100000:0.0:1789302681.681353:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:0.0:1789302681.681360:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+9:17743:x1876219371079680:12345-0@lo:104: 00010000:00010000:0.0:1789302681.681372:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.681427:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1789302681.681433:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+9:17743:x1876219371079680:12345-0@lo:104: Request processed in 74us (823us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302681.681440:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00000100:00100000:0.0:1789302681.681459:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:0.0:1789302681.681465:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+8:17743:x1876219371079808:12345-0@lo:104: 00010000:00010000:0.0:1789302681.681473:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.681525:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:0.0:1789302681.681531:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+8:17743:x1876219371079808:12345-0@lo:104: Request processed in 67us (843us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302681.681544:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00000100:00100000:0.0:1789302681.681559:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:0.0:1789302681.681564:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+7:17743:x1876219371079936:12345-0@lo:104: 00010000:00010000:0.0:1789302681.681572:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.681601:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:0.0:1789302681.681606:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+7:17743:x1876219371079936:12345-0@lo:104: Request processed in 42us (860us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302681.681611:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:0.0:1789302681.681624:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 45 00000100:00100000:0.0:1789302681.681630:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17743:x1876219371080192:12345-0@lo:104: 00010000:00010000:0.0:1789302681.681637:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.681662:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to priority blp list, 4 blwis in pool 00000100:00100000:0.0:1789302681.681666:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17743:x1876219371080192:12345-0@lo:104: Request processed in 37us (834us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302681.681672:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 45 00000100:00100000:0.0:1789302681.681683:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:0.0:1789302681.681689:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17743:x1876219371080320:12345-0@lo:104: 00010000:00010000:0.0:1789302681.681696:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.681719:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to priority blp list, 5 blwis in pool 00000100:00100000:0.0:1789302681.681723:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17743:x1876219371080320:12345-0@lo:104: Request processed in 35us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302681.681735:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00100000:0.0:1789302681.681747:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:0.0:1789302681.681753:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17743:x1876219371080448:12345-0@lo:104: 00010000:00010000:0.0:1789302681.681760:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.681789:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to priority blp list, 6 blwis in pool 00000100:00100000:0.0:1789302681.681793:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17743:x1876219371080448:12345-0@lo:104: Request processed in 42us (844us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302681.681799:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00000100:00100000:0.0:1789302681.681812:0:5828:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:0.0:1789302681.681817:0:5828:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f753c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17743:x1876219371080576:12345-0@lo:104: 00010000:00010000:0.0:1789302681.681824:0:5828:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.681849:0:5828:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/7 locks to priority blp list, 7 blwis in pool 00000100:00100000:0.0:1789302681.681854:0:5828:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f753c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17743:x1876219371080576:12345-0@lo:104: Request processed in 37us (857us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302681.681859:0:5828:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00010000:00010000:2.0:1789302681.682004:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 6 total in blp. (6 blwis in pool) 00010000:00010000:2.0:1789302681.682012:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682019:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221ee00 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302681.682023:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682031:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789302681.682038:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302681.682044:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789302681.682083:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682090:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789302681.682114:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682121:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e0221ee00/0x64fe43095184cac2 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cac9 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682132:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:2.0:1789302681.682136:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682162:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d600 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302681.682165:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682172:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789302681.682177:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.682181:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079680:0@lo:104: 10000000:01000000:2.0:1789302681.682182:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00000100:00100000:1.0:1789302681.682190:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079808:0@lo:104: 00010000:00010000:2.0:1789302681.682196:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.682197:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079936:0@lo:104: 00010000:00010000:2.0:1789302681.682202:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:1.0:1789302681.682204:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080192:0@lo:104: 00010000:00010000:2.0:1789302681.682216:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.682218:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080320:0@lo:104: 00010000:00010000:2.0:1789302681.682223:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e03c2d600/0x64fe43095184cab4 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cabb expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302681.682225:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080448:0@lo:104: 00000100:00100000:1.0:1789302681.682231:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080576:0@lo:104: 00010000:00010000:2.0:1789302681.682233:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:2.0:1789302681.682236:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682242:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221e200 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302681.682245:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682260:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789302681.682267:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302681.682272:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789302681.682298:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682303:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789302681.682309:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682315:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e0221e200/0x64fe43095184caa6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184caad expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682324:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:2.0:1789302681.682327:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682333:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb000 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302681.682335:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.682341:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789302681.682346:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302681.682357:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789302681.682370:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.682423:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00000100:00100000:3.0:1789302681.682454:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371080704:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302681.682494:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302681.682558:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371080704 00000100:00100000:3.0:1789302681.682563:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371080704 00010000:00010000:3.0:1789302681.682593:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cac2 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.682616:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789302681.682623:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371080704:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789302681.682633:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302681.682638:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cac2 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.682712:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cac2 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.682726:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f3f6fa200/0x64fe43095184cac9 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cac2 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.682738:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302681.682771:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:4acabb58-ab41-4250-a3d6-ebb667f59134+18:3303:x1876219371080704:12345-0@lo:103:ldlm_bl.0 Request processed in 148us (276us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302681.682777:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789302681.682805:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371080704:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302681.682839:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371080832:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302681.682863:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302681.682893:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302681.682898:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371080960:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302681.682918:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302681.682956:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371080832 00000100:00100000:3.0:1789302681.682960:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371080832 00010000:00010000:3.0:1789302681.682974:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cab4 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.682991:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371080960 00000100:00100000:3.0:1789302681.682994:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371080960 00010000:00010000:3.0:1789302681.683004:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184caa6 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683018:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:3.0:1789302681.683024:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371080832:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789302681.683031:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302681.683034:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cab4 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683043:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cab4 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683053:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eaa00/0x64fe43095184cabb lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cab4 expref: 19 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683061:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302681.683070:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789302681.683084:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683085:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:3300:x1876219371080832:12345-0@lo:103:ldlm_bl.0 Request processed in 62us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302681.683091:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00010000:00010000:2.0:1789302681.683092:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e011eb000/0x64fe43095184ca91 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca98 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683103:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:3.0:1789302681.683108:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3300:x1876219371080960:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.683112:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1789302681.683115:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302681.683116:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683119:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184caa6 expref: 17 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.683123:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d400 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302681.683126:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683132:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184caa6 expref: 17 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.683133:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789302681.683140:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683142:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e03c2c400/0x64fe43095184caad lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184caa6 expref: 17 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302681.683146:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302681.683151:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302681.683171:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683172:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:4acabb58-ab41-4250-a3d6-ebb667f59134+16:3300:x1876219371080960:12345-0@lo:103:ldlm_bl.0 Request processed in 65us (253us total) trans 0 rc 0/0 00010000:00010000:2.0:1789302681.683177:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00000100:00100000:3.0:1789302681.683178:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00010000:00010000:2.0:1789302681.683189:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.683196:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e03c2d400/0x64fe43095184ca75 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca7c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.683205:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1789302681.683208:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371080960:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.683208:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683215:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371080832:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.683215:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fc00 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302681.683218:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.683225:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683230:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371081088:0@lo:103:ldlm_bl.0 10000000:00010000:2.0:1789302681.683231:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302681.683236:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789302681.683248:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683252:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789302681.683254:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789302681.683262:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683266:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00010000:00010000:2.0:1789302681.683269:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e0221fc00/0x64fe43095184ca67 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca6e expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683270:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371081216:0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.683278:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302681.683284:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302681.683288:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:2.0:1789302681.683297:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302681.683304:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302681.683306:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081088 00000100:00100000:3.0:1789302681.683310:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081088 00000100:00100000:2.0:1789302681.683312:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371081344:0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789302681.683323:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca91 expref: 16 pid: 5836 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302681.683338:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302681.683339:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081216 00000100:00100000:3.0:1789302681.683341:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081216 00010000:00010000:3.0:1789302681.683350:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca75 expref: 17 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683377:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081344 00000100:00100000:3.0:1789302681.683380:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081344 00010000:00010000:3.0:1789302681.683405:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca67 expref: 18 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683417:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:3.0:1789302681.683422:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f099c2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:4acabb58-ab41-4250-a3d6-ebb667f59134+18:3302:x1876219371081088:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789302681.683429:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302681.683432:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca91 expref: 18 pid: 5836 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683440:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca91 expref: 18 pid: 5836 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.683449:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e03c2de00/0x64fe43095184ca98 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca91 expref: 18 pid: 5836 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302681.683452:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00010000:00010000:3.0:1789302681.683458:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302681.683460:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3302:x1876219371081216:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.683467:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302681.683471:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca75 expref: 17 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.683480:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca75 expref: 17 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.683482:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f099c2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3302:x1876219371081088:12345-0@lo:103:ldlm_bl.0 Request processed in 60us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302681.683487:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00010000:00010000:2.0:1789302681.683489:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb200/0x64fe43095184ca7c lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca75 expref: 17 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.683497:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302681.684694:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:3302:x1876219371081216:12345-0@lo:103:ldlm_bl.0 Request processed in 1221us (1390us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302681.684703:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:2.0:1789302681.684726:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:2.0:1789302681.684735:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:3301:x1876219371081344:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.684745:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302681.684751:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca67 expref: 14 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.684761:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca67 expref: 14 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.684788:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb400/0x64fe43095184ca6e lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca67 expref: 14 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.684799:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302681.684835:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:3301:x1876219371081344:12345-0@lo:103:ldlm_bl.0 Request processed in 103us (1496us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302681.684840:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:2.0:1789302681.684888:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371081344:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302681.684935:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371081216:0@lo:103:ldlm_bl.0 00000100:00100000:2.0:1789302681.684943:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371081088:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302681.688837:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302681.688845:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302681.688847:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302681.688863:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.688883:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:2.0:1789302681.688888:0:5832:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.688898:0:5832:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:2.0:1789302681.688905:0:5832:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302681.688911:0:5832:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:2.0:1789302681.689029:0:5832:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.689053:0:5832:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1789302681.689084:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.689091:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e03c2cc00/0x64fe43095184ca83 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184ca8a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.689109:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302681.689111:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302681.689112:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302681.689176:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302681.689195:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371081472:0@lo:103:ldlm_bl.0 00000100:00100000:1.0:1789302681.689253:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302681.689369:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081472 00000100:00100000:2.0:1789302681.689458:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081472 00010000:00010000:2.0:1789302681.689490:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca83 expref: 13 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302681.689927:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:2.0:1789302681.689938:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:3300:x1876219371081472:12345-0@lo:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.690228:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302681.690236:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184ca83 expref: 13 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.690247:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184ca83 expref: 13 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.690261:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011ebe00/0x64fe43095184ca8a lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184ca83 expref: 13 pid: 5838 timeout: 668 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.690274:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302681.690800:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+12:3300:x1876219371081472:12345-0@lo:103:ldlm_bl.0 Request processed in 862us (1544us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302681.690820:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1789302681.693360:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371081472:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302681.695645:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302681.695685:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302681.695794:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079424:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302681.695805:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371079552:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302681.695887:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246528 00000100:00100000:3.0:1789302681.695895:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246528 00010000:00010000:3.0:1789302681.695932:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 13 pid: 5838 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.695969:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246656 00000100:00100000:3.0:1789302681.695972:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246656 00010000:00010000:3.0:1789302681.695984:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 14 pid: 5836 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302681.696022:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 231 00000100:00100000:3.0:1789302681.696030:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:2348:x1876219350246528:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302681.696043:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302681.696049:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 14 pid: 5838 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.696060:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 14 pid: 5838 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.696076:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cad7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffe5 expref: 14 pid: 5838 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.696089:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302681.696132:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350246528:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 103us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302681.696139:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 231 00000100:00100000:3.0:1789302681.696154:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 232 00000100:00100000:3.0:1789302681.696172:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2347:x1876219350246656:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302681.696179:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302681.696183:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.696192:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.696201:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb600/0x64fe43095184cad0 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffde expref: 12 pid: 5836 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302681.696215:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302681.696231:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:2347:x1876219350246656:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 60us (546us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302681.696237:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 232 00000100:00100000:3.0:1789302681.700996:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302681.701091:0:17743:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17743:1876219371080064:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302681.701151:0:17743:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302681.701454:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246784 00000100:00100000:2.0:1789302681.701461:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246784 00010000:00010000:2.0:1789302681.701495:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 11 pid: 5837 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302681.701551:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 233 00000100:00100000:2.0:1789302681.701561:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:2347:x1876219350246784:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302681.701575:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302681.701582:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 11 pid: 5837 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701594:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 11 pid: 5837 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701610:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011ebc00/0x64fe43095184ca9f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f739ffd7 expref: 11 pid: 5837 timeout: 618 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701626:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701635:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701650:0:5831:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701658:0:5831:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701682:0:5831:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701700:0:5831:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701707:0:5831:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302681.701712:0:5831:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701721:0:5831:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701741:0:5831:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302681.701752:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302681.701768:0:17743:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.701776:0:17743:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302681.701787:0:17743:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011ea200/0x64fe43095184cbb0 lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17743 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302681.701808:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+10:2347:x1876219350246784:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 248us (821us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302681.701815:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 233 00010000:00010000:2.0:1789302682.253195:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302682.892763:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302682.893204:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350246912 00000100:00100000:1.0:1789302682.893218:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350246912 00000100:00100000:1.0:1789302682.893363:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 449 00000100:00100000:1.0:1789302682.893378:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+10:5588:x1876219350246912:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302682.893404:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302682.893433:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a002b expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302682.893470:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 11 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302682.893485:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 11 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302682.898793:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:1.0:1789302682.898920:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350246912:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 5539us (6166us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302682.898937:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 449 00000100:00100000:3.0:1789302682.902817:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302682.907062:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247040 00000100:00100000:0.0:1789302682.907087:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247040 00000100:00100000:0.0:1789302682.907189:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 450 00000100:00100000:0.0:1789302682.907204:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350247040:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302682.907669:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350247040:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 467us (4872us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302682.907678:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 450 00000100:00100000:3.0:1789302682.912433:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302682.912627:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247168 00000100:00100000:1.0:1789302682.912641:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247168 00000100:00100000:1.0:1789302682.912714:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:1.0:1789302682.912725:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350247168:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302682.913050:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350247168:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 326us (633us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302682.913060:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:1.0:1789302682.921314:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302682.921925:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247296 00000100:00100000:3.0:1789302682.921949:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247296 00000100:00100000:3.0:1789302682.922030:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00100000:3.0:1789302682.922043:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22993:x1876219350247296:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:3.0:1789302682.922691:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22993:x1876219350247296:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 645us (1387us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302682.922704:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00100000:0.0:1789302682.932181:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302682.932755:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247424 00000100:00100000:2.0:1789302682.932770:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247424 00000100:00100000:2.0:1789302682.932925:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 453 00000100:00100000:2.0:1789302682.932945:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22993:x1876219350247424:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302682.933298:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:22993:x1876219350247424:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 360us (1132us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302682.933309:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 453 00000100:00100000:0.0:1789302682.938732:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302682.941805:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247552 00000100:00100000:3.0:1789302682.941826:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247552 00000100:00100000:0.0:1789302682.942530:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 454 00000100:00100000:0.0:1789302682.942547:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350247552:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302682.942572:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302682.942599:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0032 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302682.942639:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302682.942653:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302682.942666:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f200, rc 0) 00000100:00100000:0.0:1789302682.942754:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350247552:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 212us (4038us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302682.942790:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 454 00000100:00100000:3.0:1789302682.946056:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302682.946378:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247680 00000100:00100000:2.0:1789302682.946391:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247680 00000100:00100000:2.0:1789302682.946449:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 455 00000100:00100000:2.0:1789302682.946460:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350247680:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302682.947433:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350247680:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 966us (1384us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302682.947445:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 455 00000100:00100000:3.0:1789302682.949640:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302682.950170:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247808 00000100:00100000:0.0:1789302682.950185:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247808 00000100:00100000:0.0:1789302682.950453:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:0.0:1789302682.950467:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350247808:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302682.951544:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350247808:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 1033us (1871us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302682.954592:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:3.0:1789302682.959155:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302682.960418:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350247936 00000100:00100000:2.0:1789302682.960435:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350247936 00000100:00100000:2.0:1789302682.961015:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:2.0:1789302682.961031:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22994:x1876219350247936:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302682.961474:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22994:x1876219350247936:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 445us (2332us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302682.961483:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:1.0:1789302682.976843:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302682.978454:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248064 00000100:00100000:2.0:1789302682.978475:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248064 00000100:00100000:2.0:1789302682.979127:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:2.0:1789302682.979141:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22994:x1876219350248064:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302682.981857:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:22994:x1876219350248064:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 2702us (5014us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302682.981884:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:2.0:1789302682.995962:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302682.997653:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248192 00000100:00100000:1.0:1789302682.997671:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248192 00000100:00100000:1.0:1789302682.997756:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:1.0:1789302682.997768:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350248192:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302682.997798:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302682.997824:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0039 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302682.997861:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 13 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302682.997884:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 13 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302682.997912:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea200, rc 0) 00000100:00100000:1.0:1789302683.000158:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350248192:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 2373us (4193us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.000173:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:0.0:1789302683.015708:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302683.016242:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248320 00000100:00100000:2.0:1789302683.016256:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248320 00000100:00100000:2.0:1789302683.016323:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 460 00000100:00100000:2.0:1789302683.016334:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350248320:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.017310:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350248320:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 969us (1606us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302683.017323:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 460 10000000:01000000:0.0:1789302683.020640:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1789302683.020651:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f120ab000 from 49 10000000:01000000:0.0:1789302683.020658:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1789302683.020727:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.020741:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.020756:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302683.020790:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371081600:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.020894:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302683.020976:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.021788:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081600 00000100:00100000:1.0:1789302683.021802:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081600 00000100:00100000:1.0:1789302683.021889:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:1.0:1789302683.021901:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+12:5841:x1876219371081600:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302683.021920:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302683.021950:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cbcc expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.021983:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cbcc expref: 13 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.021996:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbcc expref: 13 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.022006:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb600, rc 0) 00000100:00100000:1.0:1789302683.022086:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371081600:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 187us (1191us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.022092:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:2.0:1789302683.022326:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908000 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371081600:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302683.022349:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.022359:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.022445:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371081728:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.022479:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.022568:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.022676:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081728 00000100:00100000:1.0:1789302683.022682:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081728 00000100:00100000:1.0:1789302683.022718:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:1.0:1789302683.022726:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371081728:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.025365:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371081728:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 2626us (2871us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.025381:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:2.0:1789302683.025934:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371081728:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.026021:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371081856:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.026094:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.026144:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.026637:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081856 00000100:00100000:1.0:1789302683.026647:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081856 00000100:00100000:1.0:1789302683.026752:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:1.0:1789302683.026763:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371081856:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.027077:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371081856:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 315us (982us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.027085:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:2.0:1789302683.027186:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371081856:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.029442:0:17748:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17748:1876219371081984:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302683.029741:0:17748:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.029796:0:17748:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25bed540 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.029899:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371081984 00000100:00100000:0.0:1789302683.029916:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371081984 00000100:00100000:0.0:1789302683.029991:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:0.0:1789302683.030002:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17748:x1876219371081984:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302683.030353:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17748:x1876219371081984:12345-0@lo:502:llog_process_th.0 Request processed in 352us (611us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.030363:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:1.0:1789302683.030405:0:17748:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17748:1876219371081984:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302683.030484:0:17748:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17748:1876219371082112:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302683.030577:0:17748:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.030711:0:17748:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25bed540 going to sleep for 11 seconds 00000100:00100000:2.0:1789302683.030929:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371082112 00000100:00100000:2.0:1789302683.030950:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371082112 00000100:00100000:2.0:1789302683.030997:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:2.0:1789302683.031007:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17748:x1876219371082112:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302683.031606:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17748:x1876219371082112:12345-0@lo:502:llog_process_th.0 Request processed in 597us (1025us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302683.031614:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1789302683.031860:0:17748:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17748:1876219371082112:0@lo:502:llog_process_th.0 00000040:00080000:1.0:1789302683.031912:0:17748:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000020:01000000:2.0:1789302683.032176:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 49-51 (rc=0) 10000000:01000000:2.0:1789302683.032202:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:2.0:1789302683.032356:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.032370:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302683.032382:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:2.0:1789302683.032386:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:2.0:1789302683.032389:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:2.0:1789302683.032432:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.032443:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.032455:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.032479:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082240:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.032688:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.032769:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.032814:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371082240 00000100:00100000:0.0:1789302683.032823:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371082240 00000100:00100000:0.0:1789302683.033000:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:0.0:1789302683.033012:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371082240:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302683.033029:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302683.033045:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cbda expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.033074:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cbda expref: 14 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.033086:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbda expref: 14 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.033199:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f600, rc 0) 00000100:00100000:0.0:1789302683.033354:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371082240:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 343us (665us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.033361:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:2.0:1789302683.033447:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082240:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302683.033466:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.033476:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.033568:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082368:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.033610:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.033694:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371082368 00000100:00100000:1.0:1789302683.033700:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371082368 00000100:00100000:1.0:1789302683.033739:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:1.0:1789302683.033748:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371082368:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.033876:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd240 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.034087:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371082368:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 340us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.034101:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:2.0:1789302683.034307:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082368:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.034342:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082496:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.034383:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.034445:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.034581:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371082496 00000100:00100000:0.0:1789302683.034587:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371082496 00000100:00100000:0.0:1789302683.034657:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:0.0:1789302683.034667:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371082496:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.034902:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371082496:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 236us (517us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.034910:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:3.0:1789302683.035005:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082496:0@lo:503:ll_cfg_requeue.0 00000040:00080000:3.0:1789302683.035909:0:17749:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:2.0:1789302683.036028:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:2.0:1789302683.036059:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:2.0:1789302683.036072:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.036086:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302683.036098:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1789302683.036102:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f1d5a8000 from 49 10000000:01000000:2.0:1789302683.036106:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1789302683.036149:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.036159:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.036170:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.036197:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082624:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.036278:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.036343:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371082624 00000100:00100000:2.0:1789302683.036346:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.036353:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371082624 00000100:00100000:1.0:1789302683.036407:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:1.0:1789302683.036418:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371082624:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302683.036434:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302683.036452:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cbe8 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.036480:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 15 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.036495:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 15 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.036554:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:1.0:1789302683.036790:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371082624:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 373us (511us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.036799:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:2.0:1789302683.036847:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082624:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302683.036863:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.036882:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.036938:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082752:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.036964:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.036995:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.037044:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371082752 00000100:00100000:0.0:1789302683.037049:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371082752 00000100:00100000:0.0:1789302683.037080:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:0.0:1789302683.037089:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371082752:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.037567:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371082752:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 477us (600us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.037576:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1789302683.037646:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082752:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.037684:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082880:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.037728:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.037771:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.037820:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371082880 00000100:00100000:1.0:1789302683.037825:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371082880 00000100:00100000:1.0:1789302683.037859:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:1.0:1789302683.037877:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371082880:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.038057:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371082880:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 190us (329us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.038065:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:3.0:1789302683.042360:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371082880:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302683.046238:0:17750:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17750:1876219371083008:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302683.046388:0:17750:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302683.046445:0:17750:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335be600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.046564:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083008 00000100:00100000:0.0:1789302683.046574:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083008 00000100:00100000:0.0:1789302683.046630:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:0.0:1789302683.046642:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17750:x1876219371083008:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302683.047037:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17750:x1876219371083008:12345-0@lo:502:llog_process_th.0 Request processed in 396us (649us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.047047:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1789302683.047243:0:17750:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17750:1876219371083008:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302683.047316:0:17750:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17750:1876219371083136:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302683.047360:0:17750:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302683.047403:0:17750:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335be600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.047449:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083136 00000100:00100000:0.0:1789302683.047455:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083136 00000100:00100000:0.0:1789302683.047496:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:0.0:1789302683.047548:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17750:x1876219371083136:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302683.047701:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17750:x1876219371083136:12345-0@lo:502:llog_process_th.0 Request processed in 154us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.047709:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:3.0:1789302683.047880:0:17750:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17750:1876219371083136:0@lo:502:llog_process_th.0 00000040:00080000:3.0:1789302683.047905:0:17750:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000020:01000000:0.0:1789302683.050910:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 49-51 (rc=0) 10000000:01000000:0.0:1789302683.050965:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:0.0:1789302683.051006:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.051023:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302683.051038:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:0.0:1789302683.051042:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:0.0:1789302683.051046:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:0.0:1789302683.051103:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.051113:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.051127:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302683.051162:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083264:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.051257:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302683.051322:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.051475:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083264 00000100:00100000:1.0:1789302683.051493:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083264 00000100:00100000:1.0:1789302683.051598:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:1.0:1789302683.051609:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371083264:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302683.051628:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302683.051644:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cbf6 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.051675:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 16 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.051687:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 16 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.051697:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebe00, rc 0) 00000100:00100000:1.0:1789302683.051749:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371083264:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 141us (491us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.051756:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:2.0:1789302683.052026:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083264:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302683.052049:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.052060:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.052132:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083392:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.052165:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.052197:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebdf00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.052259:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083392 00000100:00100000:0.0:1789302683.052266:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083392 00000100:00100000:0.0:1789302683.052308:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:0.0:1789302683.052316:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371083392:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.052756:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371083392:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 440us (588us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.052765:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:3.0:1789302683.052893:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083392:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302683.052933:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083520:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302683.052978:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302683.053002:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335be600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.053056:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083520 00000100:00100000:0.0:1789302683.053067:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083520 00000100:00100000:0.0:1789302683.053090:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:0.0:1789302683.053097:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371083520:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.053277:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371083520:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 180us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.053284:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:3.0:1789302683.053580:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083520:0@lo:503:ll_cfg_requeue.0 00000040:00080000:3.0:1789302683.059400:0:17751:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000020:01000000:2.0:1789302683.060046:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:2.0:1789302683.060100:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:2.0:1789302683.060116:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.060130:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302683.060144:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:2.0:1789302683.060148:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f0511c000 from 49 10000000:01000000:2.0:1789302683.060152:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:2.0:1789302683.060206:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.060216:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.060228:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.060262:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083648:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.060352:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.060455:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.067207:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083648 00000100:00100000:0.0:1789302683.067233:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083648 00000100:00100000:0.0:1789302683.067327:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:0.0:1789302683.067340:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371083648:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302683.067364:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302683.067390:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc04 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.067429:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc04 expref: 17 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.067442:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc04 expref: 17 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.067453:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f400, rc 0) 00000100:00100000:0.0:1789302683.069728:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371083648:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 2379us (9362us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.069743:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:2.0:1789302683.069953:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083648:0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302683.069990:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.070007:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.070114:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083776:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.070183:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.070226:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.070399:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083776 00000100:00100000:1.0:1789302683.070410:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083776 00000100:00100000:1.0:1789302683.070467:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:1.0:1789302683.070477:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371083776:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.070883:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371083776:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 406us (698us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.070893:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:0.0:1789302683.071290:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083776:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.071351:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083904:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.071407:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302683.071445:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10ea46c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.072071:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371083904 00000100:00100000:1.0:1789302683.072082:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371083904 00000100:00100000:1.0:1789302683.072129:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:1.0:1789302683.072140:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371083904:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.072421:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371083904:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 282us (1013us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.072428:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:0.0:1789302683.072735:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371083904:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.075556:0:17752:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1500 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17752:1876219371084032:0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302683.075676:0:17752:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.075731:0:17752:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebda80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.076008:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084032 00000100:00100000:1.0:1789302683.076018:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084032 00000100:00100000:1.0:1789302683.076084:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:1.0:1789302683.076096:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17752:x1876219371084032:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302683.076460:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17752:x1876219371084032:12345-0@lo:502:llog_process_th.0 Request processed in 364us (783us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.076468:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00100000:0.0:1789302683.076651:0:17752:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1500 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17752:1876219371084032:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302683.076730:0:17752:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17752:1876219371084160:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302683.076779:0:17752:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302683.076823:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084160 00000100:00100000:0.0:1789302683.076828:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084160 00000100:00100000:0.0:1789302683.076854:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1789302683.076860:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17752:x1876219371084160:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302683.077023:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17752:x1876219371084160:12345-0@lo:502:llog_process_th.0 Request processed in 161us (242us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302683.077030:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1789302683.077083:0:17752:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10ef0a80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.077099:0:17752:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17752:1876219371084160:0@lo:502:llog_process_th.0 00000040:00080000:0.0:1789302683.077122:0:17752:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000020:01000000:1.0:1789302683.077660:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 49-51 (rc=0) 10000000:01000000:1.0:1789302683.077682:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1789302683.077695:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.077708:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302683.077718:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:1.0:1789302683.077721:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 43 10000000:01000000:1.0:1789302683.077725:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:1.0:1789302683.077763:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.077772:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.077782:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302683.077878:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084288:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.077931:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.077965:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f01254540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.077993:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084288 00000100:00100000:1.0:1789302683.078001:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084288 00000100:00100000:1.0:1789302683.078034:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 482 00000100:00100000:1.0:1789302683.078041:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371084288:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302683.078076:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302683.078092:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc12 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.078121:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc12 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.078136:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc12 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.078147:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb200, rc 0) 00000100:00100000:1.0:1789302683.084050:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371084288:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 5993us (6103us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.084068:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 482 00000100:00100000:1.0:1789302683.084184:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084288:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302683.084221:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.084237:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302683.084389:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084416:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.084466:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.084499:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f01254540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.084563:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084416 00000100:00100000:1.0:1789302683.084571:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084416 00000100:00100000:1.0:1789302683.084608:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:1.0:1789302683.084617:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371084416:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.085017:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371084416:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 399us (550us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.085025:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:1.0:1789302683.085064:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084416:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.085102:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084544:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.085147:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.085168:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b33c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.085189:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084544 00000100:00100000:1.0:1789302683.085194:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084544 00000100:00100000:1.0:1789302683.085215:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:1.0:1789302683.085221:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371084544:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.085394:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371084544:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 173us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.085402:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:1.0:1789302683.085437:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084544:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302683.085746:0:17753:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17753:1876219371084672:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302683.085786:0:17753:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.085807:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084672 00000100:00100000:1.0:1789302683.085812:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084672 00000100:00100000:1.0:1789302683.085832:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:1.0:1789302683.085838:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17753:x1876219371084672:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302683.086601:0:17753:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.089157:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17753:x1876219371084672:12345-0@lo:502:llog_process_th.0 Request processed in 3303us (3355us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.089177:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:0.0:1789302683.093850:0:17753:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17753:1876219371084672:0@lo:502:llog_process_th.0 00000020:01000000:1.0:1789302683.094808:0:17753:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:1.0:1789302683.094835:0:17753:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1789302683.094839:0:17753:0:(obd_config.c:1563:class_process_config()) marker 33 (0x1) lustre-MDT0000-m rem lustre.test_11c.lustre-OST0001_UUID 00000020:00000080:1.0:1789302683.094847:0:17753:0:(obd_config.c:1472:class_process_config()) processing cmd: ce022 00020000:01000000:1.0:1789302683.094883:0:17753:0:(lod_pool.c:649:lod_pool_remove()) lustre-OST0001_UUID removed from test_11c 00000100:00100000:1.0:1789302683.097726:0:17753:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17753:1876219371084800:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302683.097879:0:17753:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.097939:0:17753:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302683.098103:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084800 00000100:00100000:2.0:1789302683.098115:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084800 00000100:00100000:2.0:1789302683.098192:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:2.0:1789302683.098205:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17753:x1876219371084800:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302683.098592:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17753:x1876219371084800:12345-0@lo:502:llog_process_th.0 Request processed in 388us (712us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302683.098602:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000020:01000004:3.0:1789302683.098826:0:17754:0:(tgt_mount.c:153:server_get_mount()) get mount ffff960f0511c000 from lustre-MDT0000, refs=4 00000020:01000004:3.0:1789302683.098832:0:17754:0:(obd_mount.c:710:lustre_put_lsi()) put ffff960f0511c000 4 00000100:00100000:1.0:1789302683.099374:0:17753:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17753:1876219371084800:0@lo:502:llog_process_th.0 00000020:01000000:1.0:1789302683.099417:0:17753:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1789302683.099425:0:17753:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1789302683.099427:0:17753:0:(obd_config.c:1563:class_process_config()) marker 33 (0x2) lustre-MDT0000-m rem lustre.test_11c.lustre-OST0001_UUID 00000040:00080000:1.0:1789302683.099432:0:17753:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 46 count 46 00000020:01000000:2.0:1789302683.099732:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 43-45 (rc=0) 10000000:01000000:2.0:1789302683.099757:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:2.0:1789302683.099770:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.099783:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:2.0:1789302683.099797:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:2.0:1789302683.099800:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff960f050d19c0 from 1 10000000:01000000:2.0:1789302683.099814:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:2.0:1789302683.099888:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.099897:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302683.099925:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.099951:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084928:0@lo:101:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.100010:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.100055:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302683.100280:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371084928 00000100:00100000:1.0:1789302683.100290:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371084928 00000100:00100000:1.0:1789302683.100344:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 487 00000100:00100000:1.0:1789302683.100354:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371084928:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302683.100370:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302683.100387:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc20 expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.100427:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc20 expref: 19 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.100439:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc20 expref: 19 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302683.100448:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb000, rc 0) 00000100:00100000:1.0:1789302683.100685:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371084928:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 331us (673us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.100692:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 487 00000100:00100000:3.0:1789302683.103959:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371084928:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302683.104003:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302683.104019:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302683.113136:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371085056:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302683.113238:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.113301:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd900 going to sleep for 11 seconds 00000100:00100000:0.0:1789302683.113659:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085056 00000100:00100000:0.0:1789302683.113670:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085056 00000100:00100000:0.0:1789302683.113721:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 488 00000100:00100000:0.0:1789302683.113731:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371085056:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302683.114170:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371085056:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 439us (930us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302683.114179:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 488 00000100:00100000:3.0:1789302683.116883:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371085056:0@lo:501:ll_cfg_requeue.0 10000000:01000000:0.0:1789302683.117133:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:0.0:1789302683.117148:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.117161:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302683.276843:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302683.661351:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371085312:0@lo:400:kworker.0 00000100:00100000:0.0:1789302683.661469:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302683.661534:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085312 00000100:00100000:2.0:1789302683.661554:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085312 00000100:00100000:0.0:1789302683.661572:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789302683.661579:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371085184:0@lo:400:kworker.0 00000100:00100000:0.0:1789302683.661605:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302683.661624:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371085696:0@lo:400:kworker.0 00000100:00100000:1.0:1789302683.661653:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371085440:0@lo:400:kworker.0 00000100:00100000:0.0:1789302683.661654:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.661687:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302683.661711:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302683.661716:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371085568:0@lo:400:kworker.0 00000100:00100000:1.0:1789302683.661739:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789302683.661755:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302683.661777:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085696 00000100:00100000:3.0:1789302683.661782:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085696 00000100:00100000:3.0:1789302683.661822:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085440 00000100:00100000:3.0:1789302683.661825:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085440 00000100:00100000:3.0:1789302683.661850:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789302683.661872:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371085696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302683.661960:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371085696:12345-0@lo:400:kworker.0 Request processed in 101us (304us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302683.661968:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789302683.661988:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:3.0:1789302683.661995:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371085440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302683.662336:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371085440:12345-0@lo:400:kworker.0 Request processed in 340us (646us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302683.662343:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00080000:3.0:1789302683.662386:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302683.662389:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302683.662426:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371085696:0@lo:400:kworker.0 00000100:00100000:1.0:1789302683.681985:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371085440:0@lo:400:kworker.0 00000100:00100000:2.0:1789302683.684713:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085184 00000100:00100000:2.0:1789302683.684734:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085184 00000100:00100000:2.0:1789302683.684902:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085568 00000100:00100000:2.0:1789302683.684907:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085568 00000100:00100000:2.0:1789302683.684967:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:2.0:1789302683.684980:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371085312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302683.685162:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371085312:12345-0@lo:400:kworker.0 Request processed in 185us (23692us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302683.685170:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:2.0:1789302683.685190:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 323 00000100:00100000:2.0:1789302683.685197:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371085184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302683.685225:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371085184:12345-0@lo:400:kworker.0 Request processed in 30us (23620us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302683.685231:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 323 00000100:00100000:2.0:1789302683.685242:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:2.0:1789302683.685267:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371085568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302683.685304:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371085568:12345-0@lo:400:kworker.0 Request processed in 38us (23564us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302683.685310:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00080000:2.0:1789302683.685346:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302683.685392:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371085184:0@lo:400:kworker.0 00000100:00100000:2.0:1789302683.685407:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371085312:0@lo:400:kworker.0 00000100:00100000:2.0:1789302683.685476:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371085568:0@lo:400:kworker.0 00000100:00100000:3.0:1789302683.925231:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302683.925293:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302683.925325:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302683.925346:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302683.925366:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302683.925386:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302683.925411:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302683.925443:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248704 00000100:00100000:3.0:1789302683.925455:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248704 00000100:00100000:3.0:1789302683.928064:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249216 00000100:00100000:3.0:1789302683.928084:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249216 00000100:00100000:3.0:1789302683.928117:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248832 00000100:00100000:3.0:1789302683.928120:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248832 00000100:00100000:3.0:1789302683.928134:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249088 00000100:00100000:3.0:1789302683.928137:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249088 00000100:00100000:3.0:1789302683.928165:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 299 00000100:00100000:3.0:1789302683.928177:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350248704:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302683.929453:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2346:x1876219350248704:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1269us (4149us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302683.929470:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 299 00000100:00100000:3.0:1789302683.929532:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:3.0:1789302683.930335:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350249216:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302683.930427:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350249216:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 886us (5078us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302683.930437:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:3.0:1789302683.930465:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:3.0:1789302683.930475:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350248832:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302683.931192:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248448 00000100:00100000:1.0:1789302683.931206:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248448 00000100:00100000:3.0:1789302683.931662:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350248832:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (5116us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302683.931674:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:3.0:1789302683.931703:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 302 00000100:00100000:3.0:1789302683.931714:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350249088:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302683.932617:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:1.0:1789302683.932632:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350248448:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302683.932678:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350248448:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 52us (7463us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.932685:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:3.0:1789302683.932893:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350249088:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1179us (7477us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302683.932914:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 302 00000100:00100000:3.0:1789302683.933665:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248960 00000100:00100000:3.0:1789302683.933677:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248960 00000100:00100000:3.0:1789302683.933742:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350248576 00000100:00100000:3.0:1789302683.933746:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350248576 00000100:00100000:1.0:1789302683.933916:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:1.0:1789302683.933927:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2346:x1876219350248960:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302683.934671:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:3.0:1789302683.934684:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2346:x1876219350248576:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302683.934794:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2346:x1876219350248576:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 114us (9426us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302683.934803:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00080000:3.0:1789302683.934857:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302683.934874:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302683.934878:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302683.935062:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2346:x1876219350248960:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1136us (9735us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302683.935069:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:0.0:1789302684.173205:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371085824:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302684.173303:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302684.173408:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085824 00000100:00100000:0.0:1789302684.173420:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085824 00000100:00100000:0.0:1789302684.173484:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1789302684.173494:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371085824:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302684.173563:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302684.176145:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371085824:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2636us (2826us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302684.176166:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:2.0:1789302684.176744:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371085824:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302684.300870:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302684.687399:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302684.687422:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371085952:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302684.687491:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302684.687778:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371085952 00000100:00100000:0.0:1789302684.687786:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371085952 00000100:00100000:0.0:1789302684.687852:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 139 00000100:00100000:0.0:1789302684.687873:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371085952:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302684.687910:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302684.688094:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371085952:12345-0@lo:13:osp-pre-0-0.0 Request processed in 222us (602us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302684.688102:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 139 00000100:00100000:0.0:1789302684.688283:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371085952:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302685.325397:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1789302685.656899:0:17854:0:(class_obd.c:336:class_handle_ioctl()) obdclass: cmd=c00866bc len=0 uarg=00007fffea691080 20000000:00000080:2.0:1789302685.657042:0:17854:0:(mgs_handler.c:1167:mgs_iocontrol()) MGS: cmd=c00866bc len=576 karg=ffff960f3fa15c00 uarg=0000000000000000 00000040:00100000:2.0:1789302685.658111:0:17854:0:(llog.c:1072:llog_reverse_process()) index 51 : idx 48 20000000:01000000:2.0:1789302685.658119:0:17854:0:(mgs_llog.c:963:mgs_search_pool_cb()) Matched pool rec 50 cmd:0xce022 lustre.test_11c lustre-OST0001_UUID 20000000:01000000:2.0:1789302685.658132:0:17854:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-client/lustre-clilov/new lustre.test_11c fl=4 20000000:01000000:1.0:1789302685.658748:0:17855:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 42 marker 28 1->4: lustre-clilov new lustre.test_11c 20000000:01000000:1.0:1789302685.659140:0:17855:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 44 marker 28 2->4: lustre-clilov new lustre.test_11c 00000040:00080000:1.0:1789302685.659202:0:17855:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 52 count 51 00000040:00080000:2.0:1789302685.659795:0:17854:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].52, 224 off17416 20000000:01000000:2.0:1789302685.659812:0:17854:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-clilov 0xce023 lustre test_11c (null) (null) 00000040:00080000:2.0:1789302685.659844:0:17854:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].53, 112 off17528 00000040:00080000:2.0:1789302685.659893:0:17854:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x3:0x0].54, 224 off17752 20000000:01000000:2.0:1789302685.659910:0:17854:0:(mgs_llog.c:1109:mgs_modify()) modify lustre-MDT0000/lustre-MDT0000-mdtlov/new lustre.test_11c fl=4 20000000:01000000:1.0:1789302685.660452:0:17856:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 37 marker 29 1->4: lustre-MDT0000-mdtlov new lustre.test_11c 20000000:01000000:1.0:1789302685.664541:0:17856:0:(mgs_llog.c:1076:mgs_modify_handler()) Changing rec 39 marker 29 2->4: lustre-MDT0000-mdtlov new lustre.test_11c 00000040:00080000:1.0:1789302685.664737:0:17856:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 46 count 46 00000040:00080000:3.0:1789302685.665305:0:17854:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].47, 224 off16608 20000000:01000000:3.0:1789302685.665325:0:17854:0:(mgs_llog.c:1539:record_base_raw()) lcfg lustre-MDT0000-mdtlov 0xce023 lustre test_11c (null) (null) 00000040:00080000:3.0:1789302685.665361:0:17854:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].48, 120 off16728 00000040:00080000:3.0:1789302685.665411:0:17854:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0xa:0x4:0x0].49, 224 off16952 10000000:01000000:3.0:1789302685.665431:0:17854:0:(mgc_request.c:68:mgc_name2resid()) log lustre to resid 0x65727473756c/0x0 (lustre) 00010000:00010000:3.0:1789302685.665451:0:17854:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665470:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665492:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665531:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665540:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbcc expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665547:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbda expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665553:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665560:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665567:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc04 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665574:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc12 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665580:0:17854:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc20 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665587:0:17854:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 13/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665635:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cc20 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665651:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cc20 expref: 18 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.665666:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086080:0@lo:104: 00000100:00100000:3.0:1789302685.665729:0:17854:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302685.665788:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cc12 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665796:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cc12 expref: 18 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.665805:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbdf80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086208:0@lo:104: 00000100:00100000:3.0:1789302685.665824:0:17854:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302685.665864:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cc04 expref: 18 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665873:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cc04 expref: 18 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.665881:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086336:0@lo:104: 00000100:00100000:3.0:1789302685.665901:0:17854:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302685.665916:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665925:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 18 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.665945:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086464:0@lo:104: 00000100:00100000:3.0:1789302685.665967:0:17854:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302685.665984:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.665992:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 18 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.665998:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086592:0@lo:104: 00000100:00100000:3.0:1789302685.666014:0:17854:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302685.666026:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cbda expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666032:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cbda expref: 18 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.666039:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086720:0@lo:104: 00000100:00100000:3.0:1789302685.666052:0:17854:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302685.666064:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 0@lo remote: 0x64fe43095184cbcc expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666071:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 150, AT: on) ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 0@lo remote: 0x64fe43095184cbcc expref: 18 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.666077:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086848:0@lo:104: 00000100:00100000:3.0:1789302685.666093:0:17854:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302685.666107:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666126:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 12 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.666136:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086976:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302685.666299:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666306:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 12 pid: 5838 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.666314:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7d180 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371087104:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302685.666345:0:17854:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 12 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666352:0:17854:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 12 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.666359:0:17854:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371087232:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302685.666380:0:17854:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302685.666421:0:5827:0:(service.c:155:ptlrpc_grow_req_bufs()) ldlm_cbd: allocate 1 new 8192-byte reqbufs (0/2 left), rc = 0 00000100:00100000:3.0:1789302685.666432:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371086080 00000100:00100000:3.0:1789302685.666438:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371086080 00000100:00100000:3.0:1789302685.666470:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371086208 00000100:00100000:3.0:1789302685.666472:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371086208 00000100:00100000:3.0:1789302685.666482:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371086336 00000100:00100000:3.0:1789302685.666484:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371086336 00000100:00100000:3.0:1789302685.666492:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371086464 00000100:00100000:3.0:1789302685.666494:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371086464 00000100:00100000:3.0:1789302685.666548:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371086592 00000100:00100000:3.0:1789302685.666551:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371086592 00000100:00100000:3.0:1789302685.666559:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371086720 00000100:00100000:3.0:1789302685.666561:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371086720 00000100:00100000:3.0:1789302685.666568:0:5827:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371086848 00000100:00100000:3.0:1789302685.666570:0:5827:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371086848 00000100:00100000:3.0:1789302685.666582:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:3.0:1789302685.666589:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+9:17854:x1876219371086080:12345-0@lo:104: 00010000:00010000:3.0:1789302685.666601:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666654:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:3.0:1789302685.666661:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+9:17854:x1876219371086080:12345-0@lo:104: Request processed in 72us (931us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.666668:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:3.0:1789302685.666684:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:3.0:1789302685.666690:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+8:17854:x1876219371086208:12345-0@lo:104: 00010000:00010000:3.0:1789302685.666697:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666729:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to priority blp list, 2 blwis in pool 00000100:00100000:3.0:1789302685.666734:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+8:17854:x1876219371086208:12345-0@lo:104: Request processed in 44us (910us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.666739:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00000100:00100000:3.0:1789302685.666752:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:3.0:1789302685.666757:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+7:17854:x1876219371086336:12345-0@lo:104: 00010000:00010000:3.0:1789302685.666764:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666805:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/3 locks to priority blp list, 3 blwis in pool 00000100:00100000:3.0:1789302685.666810:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+7:17854:x1876219371086336:12345-0@lo:104: Request processed in 53us (909us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.666816:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000100:00100000:3.0:1789302685.666829:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:3.0:1789302685.666835:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17854:x1876219371086464:12345-0@lo:104: 00010000:00010000:3.0:1789302685.666842:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666879:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/4 locks to priority blp list, 4 blwis in pool 00000100:00100000:3.0:1789302685.666883:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+6:17854:x1876219371086464:12345-0@lo:104: Request processed in 49us (916us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.666887:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:3.0:1789302685.666898:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:3.0:1789302685.666903:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17854:x1876219371086592:12345-0@lo:104: 00010000:00010000:3.0:1789302685.666909:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666932:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/5 locks to priority blp list, 5 blwis in pool 00000100:00100000:3.0:1789302685.666937:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+5:17854:x1876219371086592:12345-0@lo:104: Request processed in 35us (922us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.666942:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000100:00100000:3.0:1789302685.666953:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:3.0:1789302685.666958:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17854:x1876219371086720:12345-0@lo:104: 00010000:00010000:3.0:1789302685.666964:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.666996:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/6 locks to priority blp list, 6 blwis in pool 00000100:00100000:3.0:1789302685.667000:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+4:17854:x1876219371086720:12345-0@lo:104: Request processed in 43us (947us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.667006:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:3.0:1789302685.667017:0:5827:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 55 00000100:00100000:3.0:1789302685.667023:0:5827:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fc48c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17854:x1876219371086848:12345-0@lo:104: 00010000:00010000:3.0:1789302685.667029:0:5827:0:(ldlm_lockd.c:2506:ldlm_callback_handler()) ### blocking ast ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667055:0:5827:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/7 locks to priority blp list, 7 blwis in pool 00000100:00100000:3.0:1789302685.667060:0:5827:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fc48c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+3:17854:x1876219371086848:12345-0@lo:104: Request processed in 38us (966us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.667065:0:5827:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 55 00010000:00010000:3.0:1789302685.667096:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 6 total in blp. (6 blwis in pool) 00010000:00010000:3.0:1789302685.667103:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667109:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:3.0:1789302685.667114:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667122:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1789302685.667130:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302685.667136:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302685.667169:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667176:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789302685.667194:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667201:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf600/0x64fe43095184cc20 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc27 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667214:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 5 total in blp. (5 blwis in pool) 00010000:00010000:3.0:1789302685.667217:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667222:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:3.0:1789302685.667225:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667231:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1789302685.667237:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302685.667242:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302685.667254:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667259:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789302685.667272:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667288:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e011eb400/0x64fe43095184cc12 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc19 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667297:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:3.0:1789302685.667300:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667306:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe800 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:3.0:1789302685.667309:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667316:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1789302685.667322:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302685.667327:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302685.667340:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667346:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789302685.667351:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667358:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f3fcbe800/0x64fe43095184cc04 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cc0b expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667366:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:3.0:1789302685.667375:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667381:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221e800 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:3.0:1789302685.667383:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667390:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1789302685.667395:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302685.667400:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302685.667413:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667418:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789302685.667424:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667431:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e0221e800/0x64fe43095184cbf6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbfd expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667439:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1789302685.667442:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667448:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf800 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:3.0:1789302685.667450:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667467:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1789302685.667474:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302685.667479:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302685.667492:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667498:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789302685.667530:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667536:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf800/0x64fe43095184cbe8 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbef expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667545:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1789302685.667548:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667553:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf000 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:3.0:1789302685.667556:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667563:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1789302685.667569:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302685.667580:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302685.667594:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667599:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789302685.667604:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667611:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960f3fcbf000/0x64fe43095184cbda lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbe1 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667621:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302685.667623:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400000000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667629:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221f800 already unused, calling callback (ffffffffc1cb2c20) 10000000:00010000:3.0:1789302685.667632:0:5833:0:(mgc_request.c:922:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1400400000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667638:0:5833:0:(ldlm_request.c:1391:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1408400000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:00010000:3.0:1789302685.667644:0:5833:0:(mgc_request.c:928:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1409400000000 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302685.667649:0:5833:0:(mgc_request.c:932:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 00010000:00010000:3.0:1789302685.667660:0:5833:0:(ldlm_request.c:1439:__ldlm_pack_lock()) ### packing ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667665:0:5833:0:(ldlm_request.c:1498:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1789302685.667677:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667682:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.203.111@tcp lock: ffff960e0221f800/0x64fe43095184cbcc lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x5c09400000020 nid: local remote: 0x64fe43095184cbd3 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.667698:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302685.667699:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302685.667701:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302685.667737:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086080:0@lo:104: 00000100:00100000:3.0:1789302685.667747:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbdf80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086208:0@lo:104: 00000100:00100000:3.0:1789302685.667754:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086336:0@lo:104: 00000100:00100000:3.0:1789302685.667762:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086464:0@lo:104: 00000100:00100000:3.0:1789302685.667769:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086592:0@lo:104: 00000100:00100000:3.0:1789302685.667776:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086720:0@lo:104: 00000100:00100000:3.0:1789302685.667782:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086848:0@lo:104: 00010000:00010000:3.0:1789302685.667809:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302685.667825:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371087616:0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302685.668463:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302685.668487:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371087360:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.668556:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302685.668591:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371087872:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.668615:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302685.668637:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302685.668643:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371087744:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.668663:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302685.668693:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371087488:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.668715:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302685.668731:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371088000:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.668750:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302685.668778:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371087360 00000100:00100000:0.0:1789302685.668785:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371087360 00010000:00010000:0.0:1789302685.668816:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cc20 expref: 19 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302685.668835:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371087872 00000100:00100000:0.0:1789302685.668838:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371087872 00010000:00010000:0.0:1789302685.668876:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 20 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302685.668888:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371087744 00000100:00100000:0.0:1789302685.668891:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371087744 00010000:00010000:0.0:1789302685.668902:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 21 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302685.668912:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371087488 00000100:00100000:0.0:1789302685.668914:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371087488 00010000:00010000:0.0:1789302685.668932:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cc12 expref: 22 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302685.668942:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088000 00000100:00100000:0.0:1789302685.668944:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088000 00010000:00010000:0.0:1789302685.668953:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbda expref: 23 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302685.668965:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:0.0:1789302685.668972:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+23:3301:x1876219371087360:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302685.668984:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302685.668990:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cc20 expref: 23 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669002:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cc20 expref: 23 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669017:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb000/0x64fe43095184cc27 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cc20 expref: 23 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669032:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302685.669065:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b2850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+22:3301:x1876219371087360:12345-0@lo:103:ldlm_bl.0 Request processed in 94us (509us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302685.669078:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:0.0:1789302685.669125:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:0.0:1789302685.669132:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+21:3301:x1876219371087872:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302685.669139:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302685.669144:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 21 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669153:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 21 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669162:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eaa00/0x64fe43095184cbef lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cbe8 expref: 21 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669170:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302685.669191:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+20:3301:x1876219371087872:12345-0@lo:103:ldlm_bl.0 Request processed in 61us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302685.669198:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:0.0:1789302685.669211:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:0.0:1789302685.669216:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371087744:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302685.669223:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302685.669227:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 19 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669234:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 19 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669244:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011ebe00/0x64fe43095184cbfd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cbf6 expref: 19 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669262:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302685.669285:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:3301:x1876219371087744:12345-0@lo:103:ldlm_bl.0 Request processed in 69us (622us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302685.669291:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:0.0:1789302685.669304:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789302685.669309:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:3302:x1876219371087488:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302685.669316:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302685.669320:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cc12 expref: 17 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669329:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cc12 expref: 17 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669338:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb200/0x64fe43095184cc19 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cc12 expref: 17 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669346:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302685.669373:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:3302:x1876219371087488:12345-0@lo:103:ldlm_bl.0 Request processed in 63us (658us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302685.669378:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789302685.669391:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1789302685.669397:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:3302:x1876219371088000:12345-0@lo:103:ldlm_bl.0 00010000:00010000:0.0:1789302685.669404:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302685.669408:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbda expref: 15 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669417:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cbda expref: 15 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669435:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e0221f600/0x64fe43095184cbe1 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cbda expref: 15 pid: 5838 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302685.669444:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302685.669467:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:3302:x1876219371088000:12345-0@lo:103:ldlm_bl.0 Request processed in 70us (716us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302685.669473:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1789302685.669549:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371087488:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.669557:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371088000:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.669585:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371087744:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.669592:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371087360:0@lo:103:ldlm_bl.0 00000100:00100000:0.0:1789302685.669599:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7ca300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371087872:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302685.672355:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302685.672410:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371088128:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302685.672446:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302685.672801:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302685.672876:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371086976:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302685.674077:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371087616 00000100:00100000:1.0:1789302685.674087:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371087616 00010000:00010000:1.0:1789302685.674137:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cc04 expref: 14 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302685.674201:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088128 00000100:00100000:1.0:1789302685.674205:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088128 00010000:00010000:1.0:1789302685.674218:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbcc expref: 15 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302685.674279:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249344 00000100:00100000:0.0:1789302685.674284:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249344 00010000:00010000:0.0:1789302685.674303:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 13 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.674620:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:3.0:1789302685.674630:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:3303:x1876219371087616:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789302685.674644:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302685.674661:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cc04 expref: 15 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674674:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cc04 expref: 15 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674690:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e0221f400/0x64fe43095184cc0b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cc04 expref: 15 pid: 5837 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674701:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302685.674754:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+14:3303:x1876219371087616:12345-0@lo:103:ldlm_bl.0 Request processed in 126us (2407us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.674760:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:3.0:1789302685.674775:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:3.0:1789302685.674780:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+13:3303:x1876219371088128:12345-0@lo:103:ldlm_bl.0 00010000:00010000:3.0:1789302685.674787:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302685.674790:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x60000400000020 nid: 0@lo remote: 0x64fe43095184cbcc expref: 13 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674800:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000400000020 nid: 0@lo remote: 0x64fe43095184cbcc expref: 13 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674809:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb600/0x64fe43095184cbd3 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x44801400000020 nid: 0@lo remote: 0x64fe43095184cbcc expref: 13 pid: 5836 timeout: 672 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674816:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302685.674832:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:4acabb58-ab41-4250-a3d6-ebb667f59134+12:3303:x1876219371088128:12345-0@lo:103:ldlm_bl.0 Request processed in 53us (2386us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.674838:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:3.0:1789302685.674858:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 241 00000100:00100000:3.0:1789302685.674870:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350249344:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302685.674878:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302685.674883:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 13 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674891:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 13 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674901:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011ea200/0x64fe43095184cbc5 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0039 expref: 13 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.674908:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302685.675160:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b2450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2347:x1876219350249344:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 289us (2359us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.675167:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 241 00000100:00100000:3.0:1789302685.675230:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371087616:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302685.675238:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371088128:0@lo:103:ldlm_bl.0 00000100:00100000:3.0:1789302685.675283:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7d180 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371087104:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302685.675292:0:17854:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job lctl:MGS:17854:1876219371087232:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302685.675332:0:17854:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 3/0,1 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302685.677567:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302685.677597:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302685.677671:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249472 00000100:00100000:3.0:1789302685.677680:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249472 00010000:00010000:3.0:1789302685.677712:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 12 pid: 5838 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.677753:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249600 00000100:00100000:3.0:1789302685.677756:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249600 00010000:00010000:3.0:1789302685.677767:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 13 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302685.677783:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:3.0:1789302685.677790:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350249472:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302685.677804:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302685.677809:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 13 pid: 5838 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.677820:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 13 pid: 5838 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.677835:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e0221f200/0x64fe43095184cbbe lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0032 expref: 13 pid: 5838 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.677861:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302685.677904:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:2348:x1876219350249472:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 114us (345us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.677910:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:3.0:1789302685.677935:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:3.0:1789302685.677942:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:2345:x1876219350249600:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302685.677949:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302685.677953:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 4/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 11 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.677972:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 11 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.677982:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e011eb800/0x64fe43095184cbb7 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a002b expref: 11 pid: 5836 timeout: 622 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.677994:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 3/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678002:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 4/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678017:0:5829:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678024:0:5829:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 5/0,1 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678032:0:5829:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 4/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678039:0:5829:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678046:0:5829:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3f6f7e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302685.678050:0:5829:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 5/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678058:0:5829:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 6/0,0 mode: EX/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678085:0:5829:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 4/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678097:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302685.678118:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+10:2345:x1876219350249600:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 176us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302685.678124:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 243 00010000:00010000:3.0:1789302685.678151:0:17854:0:(ldlm_request.c:190:ldlm_completion_tail()) ### client-side enqueue: destroyed ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678159:0:17854:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 1/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302685.678164:0:17854:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc2e lrc: 0/0,0 mode: --/EX res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 17854 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302686.349185:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 10000000:01000000:0.0:1789302687.180712:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:0.0:1789302687.180723:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f120ab000 from 52 10000000:01000000:0.0:1789302687.180730:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:0.0:1789302687.180800:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221f600/0x64fe43095184cc35 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.180814:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221f600/0x64fe43095184cc35 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.180863:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221f600/0x64fe43095184cc35 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302687.180898:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088256:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.181153:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302687.181334:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088256 00000100:00100000:2.0:1789302687.181345:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088256 00000100:00100000:0.0:1789302687.181430:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb66c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.183207:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:2.0:1789302687.183238:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+12:5841:x1876219371088256:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302687.183262:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302687.183310:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cc3c lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc35 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.183349:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cc3c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc35 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.183363:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3fcbe000/0x64fe43095184cc3c lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc35 expref: 13 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.183374:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe000, rc 0) 00000100:00100000:2.0:1789302687.183484:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371088256:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 265us (2331us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.183499:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:0.0:1789302687.183645:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088256:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302687.183672:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221f600/0x64fe43095184cc35 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.183684:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221f600/0x64fe43095184cc35 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302687.183785:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088384:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.183828:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.195087:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb66c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.195637:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088384 00000100:00100000:1.0:1789302687.195651:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088384 00000100:00100000:1.0:1789302687.195726:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:1.0:1789302687.195738:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371088384:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.196209:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371088384:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 472us (12380us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.196219:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:0.0:1789302687.199556:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088384:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.199668:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088512:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.199777:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.199831:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb66c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.200325:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088512 00000100:00100000:2.0:1789302687.200336:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088512 00000100:00100000:2.0:1789302687.200410:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1789302687.200421:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371088512:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302687.200951:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371088512:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 530us (1172us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.200962:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:0.0:1789302687.201197:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088512:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.201759:0:17857:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17857:1876219371088640:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.201814:0:17857:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.201861:0:17857:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.201925:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088640 00000100:00100000:1.0:1789302687.201932:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088640 00000100:00100000:2.0:1789302687.202017:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789302687.202028:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17857:x1876219371088640:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302687.202542:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17857:x1876219371088640:12345-0@lo:502:llog_process_th.0 Request processed in 515us (726us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.202550:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:0.0:1789302687.202662:0:17857:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17857:1876219371088640:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.202717:0:17857:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17857:1876219371088768:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.202757:0:17857:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.202790:0:17857:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.202894:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088768 00000100:00100000:1.0:1789302687.202901:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088768 00000100:00100000:1.0:1789302687.202951:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:1.0:1789302687.202959:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17857:x1876219371088768:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302687.203127:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:17857:x1876219371088768:12345-0@lo:502:llog_process_th.0 Request processed in 169us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.203135:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:0.0:1789302687.203397:0:17857:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17857:1876219371088768:0@lo:502:llog_process_th.0 00000040:00080000:0.0:1789302687.203435:0:17857:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 55 count 54 00000020:01000000:1.0:1789302687.203894:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-54 (rc=0) 10000000:01000000:1.0:1789302687.204050:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:1.0:1789302687.204064:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221f600/0x64fe43095184cc35 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.204078:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221f600/0x64fe43095184cc35 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 2 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc3c expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302687.204089:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0001 10000000:01000000:1.0:1789302687.204091:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0001-0000000000000000 from 11 10000000:01000000:1.0:1789302687.204095:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0001 (res 0x65727473756c) 00010000:00010000:1.0:1789302687.204133:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011eae00/0x64fe43095184cc43 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.204143:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e011eae00/0x64fe43095184cc43 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.204154:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e011eae00/0x64fe43095184cc43 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302687.204174:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088896:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.204224:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302687.204286:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335bea80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.204357:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371088896 00000100:00100000:2.0:1789302687.204364:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371088896 00000100:00100000:2.0:1789302687.204419:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:2.0:1789302687.204438:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+13:5841:x1876219371088896:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302687.204453:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302687.204468:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3fcbfa00/0x64fe43095184cc4a lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc43 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.204531:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3fcbfa00/0x64fe43095184cc4a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc43 expref: 14 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.204544:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3fcbfa00/0x64fe43095184cc4a lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc43 expref: 14 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.204554:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfa00, rc 0) 00000100:00100000:2.0:1789302687.204600:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371088896:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 165us (376us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.204730:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:0.0:1789302687.205068:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371088896:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302687.205088:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e011eae00/0x64fe43095184cc43 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.205098:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e011eae00/0x64fe43095184cc43 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302687.205187:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089024:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.205222:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.205247:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.205438:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089024 00000100:00100000:1.0:1789302687.205444:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089024 00000100:00100000:1.0:1789302687.205473:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 496 00000100:00100000:1.0:1789302687.205481:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371089024:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.207351:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371089024:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 1860us (2117us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.207367:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 496 00000100:00100000:0.0:1789302687.207992:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089024:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.208064:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089152:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.208131:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.208163:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.208460:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089152 00000100:00100000:1.0:1789302687.208470:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089152 00000100:00100000:1.0:1789302687.208699:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:1.0:1789302687.208710:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371089152:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.209239:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371089152:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 529us (1105us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.209249:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:0.0:1789302687.209376:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089152:0@lo:503:ll_cfg_requeue.0 00000040:00080000:0.0:1789302687.217319:0:17858:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x9:0x0] index 64768 count 11 00000020:01000000:1.0:1789302687.218111:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0001 gen 11-10 (rc=0) 10000000:01000000:1.0:1789302687.218142:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0001' succeeded (0). 00010000:00010000:1.0:1789302687.218155:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011eae00/0x64fe43095184cc43 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.218170:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e011eae00/0x64fe43095184cc43 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc4a expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:1.0:1789302687.218182:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:1.0:1789302687.218186:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f1d5a8000 from 52 10000000:01000000:1.0:1789302687.218189:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:1.0:1789302687.218245:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184cc51 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.218278:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184cc51 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.218293:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184cc51 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302687.218326:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089280:0@lo:101:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.218422:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302687.218478:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335bed80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302687.218715:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089280 00000100:00100000:3.0:1789302687.218725:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089280 00000100:00100000:3.0:1789302687.218786:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:3.0:1789302687.218797:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+14:5841:x1876219371089280:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302687.219286:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302687.219307:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc58 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc51 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.219341:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc58 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc51 expref: 15 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.219354:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3f6f7e00/0x64fe43095184cc58 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc51 expref: 15 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.219376:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6f7e00, rc 0) 00000100:00100000:3.0:1789302687.219465:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371089280:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 669us (1042us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.219473:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00100000:1.0:1789302687.220276:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089280:0@lo:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302687.220302:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184cc51 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.220314:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184cc51 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302687.220424:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089408:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.220474:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302687.220562:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335bed80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.220716:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089408 00000100:00100000:2.0:1789302687.220726:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089408 00000100:00100000:2.0:1789302687.220783:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:2.0:1789302687.220794:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371089408:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302687.221346:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371089408:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 552us (869us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.221356:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:1.0:1789302687.221640:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089408:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.221693:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089536:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.221748:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302687.221784:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335bed80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302687.221917:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089536 00000100:00100000:3.0:1789302687.221923:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089536 00000100:00100000:3.0:1789302687.221962:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 500 00000100:00100000:3.0:1789302687.221970:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371089536:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.222173:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371089536:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 203us (425us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.222180:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 500 00000100:00100000:1.0:1789302687.222419:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089536:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.225888:0:17859:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17859:1876219371089664:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.226020:0:17859:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.226090:0:17859:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6d80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.226266:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089664 00000100:00100000:2.0:1789302687.226275:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089664 00000100:00100000:2.0:1789302687.226336:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 501 00000100:00100000:2.0:1789302687.226348:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17859:x1876219371089664:12345-0@lo:502:llog_process_th.0 00000100:00100000:2.0:1789302687.226818:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17859:x1876219371089664:12345-0@lo:502:llog_process_th.0 Request processed in 471us (798us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.226850:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 501 00000100:00100000:0.0:1789302687.228097:0:17859:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17859:1876219371089664:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.228205:0:17859:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17859:1876219371089792:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.228280:0:17859:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302687.228435:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089792 00000100:00100000:1.0:1789302687.228452:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089792 00000100:00100000:0.0:1789302687.233125:0:17859:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6d80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.234134:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:1.0:1789302687.234161:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17859:x1876219371089792:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302687.234846:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+15:17859:x1876219371089792:12345-0@lo:502:llog_process_th.0 Request processed in 688us (6554us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.234857:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:0.0:1789302687.236443:0:17859:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17859:1876219371089792:0@lo:502:llog_process_th.0 00000040:00080000:0.0:1789302687.236895:0:17859:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 55 count 54 00000020:01000000:3.0:1789302687.237274:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-54 (rc=0) 10000000:01000000:3.0:1789302687.237300:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1789302687.237315:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184cc51 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.237330:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e011ea600/0x64fe43095184cc51 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 4 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc58 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302687.237342:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-OST0000 10000000:01000000:3.0:1789302687.237345:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-OST0000-0000000000000000 from 14 10000000:01000000:3.0:1789302687.237348:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-OST0000 (res 0x65727473756c) 00010000:00010000:3.0:1789302687.237400:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3f6f7c00/0x64fe43095184cc5f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.237423:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3f6f7c00/0x64fe43095184cc5f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.237457:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3f6f7c00/0x64fe43095184cc5f lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302687.237488:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089920:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.237885:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.238163:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.242117:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371089920 00000100:00100000:2.0:1789302687.242138:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371089920 00000100:00100000:2.0:1789302687.242239:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1789302687.242252:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+15:5841:x1876219371089920:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:2.0:1789302687.242275:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302687.242302:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3fcbf000/0x64fe43095184cc66 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc5f expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.242338:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3fcbf000/0x64fe43095184cc66 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc5f expref: 16 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.242352:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3fcbf000/0x64fe43095184cc66 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc5f expref: 16 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302687.242362:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf000, rc 0) 00000100:00100000:2.0:1789302687.242469:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371089920:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 219us (4584us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.242476:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:3.0:1789302687.246663:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371089920:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302687.246710:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3f6f7c00/0x64fe43095184cc5f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc66 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.246727:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3f6f7c00/0x64fe43095184cc5f lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc66 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302687.246876:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090048:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.247285:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.247346:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:0.0:1789302687.247418:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090048 00000100:00100000:0.0:1789302687.247428:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090048 00000100:00100000:0.0:1789302687.247499:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:0.0:1789302687.247534:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371090048:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.247922:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371090048:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 391us (637us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.247930:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:3.0:1789302687.248335:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090048:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.248389:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090176:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.249358:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.250111:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.251199:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090176 00000100:00100000:2.0:1789302687.251215:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090176 00000100:00100000:2.0:1789302687.251276:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1789302687.251288:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371090176:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302687.251632:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371090176:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 346us (2277us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.251640:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000100:00100000:3.0:1789302687.253149:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090176:0@lo:503:ll_cfg_requeue.0 00000040:00080000:1.0:1789302687.258345:0:17860:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x8:0x0] index 64768 count 14 00000020:01000000:3.0:1789302687.266948:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-OST0000 gen 14-13 (rc=0) 10000000:01000000:3.0:1789302687.267206:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-OST0000' succeeded (0). 00010000:00010000:3.0:1789302687.267221:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3f6f7c00/0x64fe43095184cc5f lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc66 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.267247:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3f6f7c00/0x64fe43095184cc5f lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc66 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302687.267261:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-client 10000000:01000000:3.0:1789302687.267264:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-client-ffff960f0511c000 from 52 10000000:01000000:3.0:1789302687.267268:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c) 00010000:00010000:3.0:1789302687.267315:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3f6fa200/0x64fe43095184cc6d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.267324:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3f6fa200/0x64fe43095184cc6d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.267338:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3f6fa200/0x64fe43095184cc6d lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302687.267369:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090304:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.267456:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.267680:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302687.267689:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090304 00000100:00100000:2.0:1789302687.267708:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090304 00000100:00100000:3.0:1789302687.267928:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:3.0:1789302687.267940:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+16:5841:x1876219371090304:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302687.267957:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302687.267971:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3f6fa800/0x64fe43095184cc74 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc6d expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.268060:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3f6fa800/0x64fe43095184cc74 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc6d expref: 17 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.268073:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3f6fa800/0x64fe43095184cc74 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc6d expref: 17 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.268090:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3f6fa800, rc 0) 00000100:00100000:3.0:1789302687.268146:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371090304:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 208us (692us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.268153:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:0.0:1789302687.268447:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9880 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090304:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302687.268470:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3f6fa200/0x64fe43095184cc6d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc74 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.268482:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3f6fa200/0x64fe43095184cc6d lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc74 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302687.270186:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090432:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.270271:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.270324:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090432 00000100:00100000:0.0:1789302687.270333:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090432 00000100:00100000:0.0:1789302687.270390:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:0.0:1789302687.270428:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371090432:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.271800:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6e40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.271849:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090432:0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.271922:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090560:0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.272091:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302687.272120:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335bea80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302687.273284:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090560 00000100:00100000:3.0:1789302687.273297:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090560 00000100:00100000:3.0:1789302687.273358:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789302687.273370:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371090560:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.273758:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371090432:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 973us (1128us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.273771:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:3.0:1789302687.274125:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371090560:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 756us (2145us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.274132:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789302687.274194:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090560:0@lo:503:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.274470:0:17862:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17862:1876219371090688:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302687.278126:0:17862:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.278180:0:17862:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e840 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.282957:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090688 00000100:00100000:1.0:1789302687.282981:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090688 00000100:00100000:1.0:1789302687.283063:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:1.0:1789302687.283076:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17862:x1876219371090688:12345-0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302687.285049:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17862:x1876219371090688:12345-0@lo:502:llog_process_th.0 Request processed in 1967us (6926us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.285066:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1789302687.286915:0:17862:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17862:1876219371090688:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302687.287034:0:17862:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17862:1876219371090816:0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302687.287134:0:17862:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.287184:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090816 00000100:00100000:3.0:1789302687.287192:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090816 00000100:00100000:3.0:1789302687.287251:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1789302687.287261:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17862:x1876219371090816:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302687.287645:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+17:17862:x1876219371090816:12345-0@lo:502:llog_process_th.0 Request processed in 383us (509us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.287661:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1789302687.287719:0:17862:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302687.287733:0:17862:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17862:1876219371090816:0@lo:502:llog_process_th.0 00000040:00080000:3.0:1789302687.287754:0:17862:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x3:0x0] index 55 count 54 00000020:01000000:3.0:1789302687.287833:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-client gen 52-54 (rc=0) 10000000:01000000:3.0:1789302687.287871:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-client' succeeded (0). 00010000:00010000:3.0:1789302687.287883:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3f6fa200/0x64fe43095184cc6d lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc74 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.287896:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3f6fa200/0x64fe43095184cc6d lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 6 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc74 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:3.0:1789302687.287908:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-MDT0000 10000000:01000000:3.0:1789302687.287911:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 46 10000000:01000000:3.0:1789302687.287915:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c) 00010000:00010000:3.0:1789302687.287948:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3f6fae00/0x64fe43095184cc7b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.287958:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960f3f6fae00/0x64fe43095184cc7b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.287970:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960f3f6fae00/0x64fe43095184cc7b lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302687.287987:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090944:0@lo:101:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.288020:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.288037:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302687.288052:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371090944 00000100:00100000:3.0:1789302687.288063:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371090944 00000100:00100000:3.0:1789302687.288090:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:3.0:1789302687.288096:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+17:5841:x1876219371090944:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302687.288106:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302687.288124:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f120dcc00/0x64fe43095184cc82 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc7b expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.288151:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f120dcc00/0x64fe43095184cc82 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc7b expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.288162:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f120dcc00/0x64fe43095184cc82 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc7b expref: 18 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.288172:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dcc00, rc 0) 00000100:00100000:3.0:1789302687.288199:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371090944:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 104us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.288205:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00000100:00100000:3.0:1789302687.288234:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371090944:0@lo:101:ll_cfg_requeue.0 00010000:00010000:3.0:1789302687.288247:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960f3f6fae00/0x64fe43095184cc7b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc82 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.288256:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960f3f6fae00/0x64fe43095184cc7b lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc82 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302687.288297:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091072:0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.288319:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302687.288331:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302687.288343:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091072 00000100:00100000:3.0:1789302687.288354:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091072 00000100:00100000:3.0:1789302687.288376:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:3.0:1789302687.288382:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371091072:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:3.0:1789302687.291988:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371091072:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 3592us (3654us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.292008:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:2.0:1789302687.292290:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091072:0@lo:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302687.292369:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091200:0@lo:503:ll_cfg_requeue.0 00000100:00100000:2.0:1789302687.292451:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302687.292523:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05c00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.292910:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091200 00000100:00100000:1.0:1789302687.292920:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091200 00000100:00100000:1.0:1789302687.292970:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:1.0:1789302687.292981:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371091200:12345-0@lo:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.293218:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371091200:12345-0@lo:503:ll_cfg_requeue.0 Request processed in 239us (767us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.293227:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1789302687.302143:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091200:0@lo:503:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.304227:0:17863:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17863:1876219371091328:0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.304340:0:17863:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.304415:0:17863:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe803c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302687.304481:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091328 00000100:00100000:3.0:1789302687.304527:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091328 00000100:00100000:3.0:1789302687.304591:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 514 00000100:00100000:3.0:1789302687.304786:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17863:x1876219371091328:12345-0@lo:502:llog_process_th.0 00000100:00100000:3.0:1789302687.305157:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17863:x1876219371091328:12345-0@lo:502:llog_process_th.0 Request processed in 554us (815us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302687.305166:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 514 00000100:00100000:0.0:1789302687.305350:0:17863:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17863:1876219371091328:0@lo:502:llog_process_th.0 00000020:01000000:0.0:1789302687.305386:0:17863:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1 00000020:00000080:0.0:1789302687.305411:0:17863:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:0.0:1789302687.305414:0:17863:0:(obd_config.c:1563:class_process_config()) marker 35 (0x1) lustre-MDT0000-m del lustre.test_11c 00000020:00000080:0.0:1789302687.305421:0:17863:0:(obd_config.c:1472:class_process_config()) processing cmd: ce023 00000100:00100000:1.0:1789302687.313818:0:17863:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17863:1876219371091456:0@lo:502:llog_process_th.0 00000100:00100000:1.0:1789302687.313944:0:17863:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302687.314018:0:17863:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f01254300 going to sleep for 11 seconds 00000100:00100000:0.0:1789302687.314205:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091456 00000100:00100000:0.0:1789302687.314216:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091456 00000100:00100000:0.0:1789302687.314278:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 515 00000100:00100000:0.0:1789302687.314289:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17863:x1876219371091456:12345-0@lo:502:llog_process_th.0 00000100:00100000:0.0:1789302687.314573:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+18:17863:x1876219371091456:12345-0@lo:502:llog_process_th.0 Request processed in 285us (628us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.314581:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 515 00000100:00100000:1.0:1789302687.315944:0:17863:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job llog_process_th:4acabb58-ab41-4250-a3d6-ebb667f59134:17863:1876219371091456:0@lo:502:llog_process_th.0 00000020:01000000:1.0:1789302687.316007:0:17863:0:(obd_config.c:1893:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2 00000020:00000080:1.0:1789302687.316018:0:17863:0:(obd_config.c:1472:class_process_config()) processing cmd: cf010 00000020:00000080:1.0:1789302687.316021:0:17863:0:(obd_config.c:1563:class_process_config()) marker 35 (0x2) lustre-MDT0000-m del lustre.test_11c 00000040:00080000:1.0:1789302687.316025:0:17863:0:(llog.c:862:llog_process_thread()) stop processing plain [0xa:0x4:0x0] index 50 count 49 00000020:01000000:0.0:1789302687.317058:0:5841:0:(obd_config.c:2145:class_config_parse_llog()) Processed log lustre-MDT0000 gen 46-49 (rc=0) 10000000:01000000:0.0:1789302687.317093:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-MDT0000' succeeded (0). 00010000:00010000:0.0:1789302687.317106:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960f3f6fae00/0x64fe43095184cc7b lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc82 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.317122:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960f3f6fae00/0x64fe43095184cc7b lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 7 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc82 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 10000000:01000000:0.0:1789302687.317157:0:5841:0:(mgc_request.c:611:do_requeue()) updating log lustre-sptlrpc 10000000:01000000:0.0:1789302687.317161:0:5841:0:(mgc_request.c:1802:mgc_process_log()) Process log lustre-sptlrpc-ffff960f050d19c0 from 1 10000000:01000000:0.0:1789302687.317164:0:5841:0:(mgc_request.c:996:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c) 00010000:00010000:0.0:1789302687.317212:0:5841:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221ea00/0x64fe43095184cc89 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.317221:0:5841:0:(ldlm_request.c:1107:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000000000000 ns: MGC192.168.203.111@tcp lock: ffff960e0221ea00/0x64fe43095184cc89 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.317232:0:5841:0:(ldlm_request.c:1194:ldlm_cli_enqueue()) ### sending request ns: MGC192.168.203.111@tcp lock: ffff960e0221ea00/0x64fe43095184cc89 lrc: 3/1,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302687.317261:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091584:0@lo:101:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.317347:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.317379:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80900 going to sleep for 11 seconds 00000100:00100000:0.0:1789302687.333376:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091584 00000100:00100000:0.0:1789302687.333400:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091584 00000100:00100000:0.0:1789302687.333482:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 516 00000100:00100000:0.0:1789302687.333541:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+18:5841:x1876219371091584:12345-0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302687.333567:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302687.333593:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e0221f000/0x64fe43095184cc90 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: local remote: 0x64fe43095184cc89 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.333628:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e0221f000/0x64fe43095184cc90 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x50000000000000 nid: 0@lo remote: 0x64fe43095184cc89 expref: 19 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.333642:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e0221f000/0x64fe43095184cc90 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x40000000000000 nid: 0@lo remote: 0x64fe43095184cc89 expref: 19 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.333666:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f000, rc 0) 00000100:00100000:0.0:1789302687.333765:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371091584:12345-0@lo:101:ll_cfg_requeue.0 Request processed in 274us (16418us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.333772:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 516 00000100:00100000:0.0:1789302687.333858:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091584:0@lo:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302687.333881:0:5841:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGC192.168.203.111@tcp lock: ffff960e0221ea00/0x64fe43095184cc89 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc90 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.333892:0:5841:0:(ldlm_request.c:841:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: MGC192.168.203.111@tcp lock: ffff960e0221ea00/0x64fe43095184cc89 lrc: 4/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x1000000000000 nid: local remote: 0x64fe43095184cc90 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302687.333993:0:5841:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091712:0@lo:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.334031:0:5841:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302687.334074:0:5841:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80900 going to sleep for 11 seconds 00000100:00100000:1.0:1789302687.334782:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091712 00000100:00100000:1.0:1789302687.334793:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091712 00000100:00100000:1.0:1789302687.334849:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:1.0:1789302687.334858:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371091712:12345-0@lo:501:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.335156:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:5841:x1876219371091712:12345-0@lo:501:ll_cfg_requeue.0 Request processed in 297us (1123us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302687.335164:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:1.0:1789302687.335228:0:5841:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_cfg_requeue:4acabb58-ab41-4250-a3d6-ebb667f59134:5841:1876219371091712:0@lo:501:ll_cfg_requeue.0 10000000:01000000:1.0:1789302687.335270:0:5841:0:(mgc_request.c:1879:mgc_process_log()) MGC192.168.203.111@tcp: configuration from log 'lustre-sptlrpc' failed (-2). 00010000:00010000:1.0:1789302687.335281:0:5841:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.203.111@tcp lock: ffff960e0221ea00/0x64fe43095184cc89 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc90 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.335292:0:5841:0:(ldlm_lock.c:919:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: MGC192.168.203.111@tcp lock: ffff960e0221ea00/0x64fe43095184cc89 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 8 type: PLN flags: 0x11000000000000 nid: local remote: 0x64fe43095184cc90 expref: -99 pid: 5841 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302687.373234:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302687.565682:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302687.565849:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249728 00000100:00100000:0.0:1789302687.565863:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249728 00000100:00100000:0.0:1789302687.565945:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 518 00000100:00100000:0.0:1789302687.565958:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+10:5588:x1876219350249728:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302687.565981:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302687.566005:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e0221ec00/0x64fe43095184cc97 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0040 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.566044:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e0221ec00/0x64fe43095184cc97 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0040 expref: 11 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.566058:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e0221ec00/0x64fe43095184cc97 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0040 expref: 11 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.566088:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221ec00, rc 0) 00000100:00100000:0.0:1789302687.566171:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350249728:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 216us (505us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.566179:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 518 00000100:00100000:2.0:1789302687.576154:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302687.576265:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249856 00000100:00100000:0.0:1789302687.576280:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249856 00000100:00100000:0.0:1789302687.576357:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 519 00000100:00100000:0.0:1789302687.576369:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350249856:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:0.0:1789302687.577654:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350249856:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 1276us (1504us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.577670:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 519 00000100:00100000:2.0:1789302687.585256:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302687.585362:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350249984 00000100:00100000:1.0:1789302687.585376:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350249984 00000100:00100000:1.0:1789302687.585444:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 520 00000100:00100000:1.0:1789302687.585456:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350249984:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.585848:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350249984:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 393us (606us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.585857:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 520 00000100:00100000:3.0:1789302687.589041:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302687.589124:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350250112 00000100:00100000:0.0:1789302687.589136:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350250112 00000100:00100000:0.0:1789302687.589198:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 521 00000100:00100000:0.0:1789302687.589209:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:23013:x1876219350250112:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:0.0:1789302687.596742:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:23013:x1876219350250112:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 7515us (7695us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.596763:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 521 00000100:00100000:3.0:1789302687.608469:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302687.608689:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350250240 00000100:00100000:2.0:1789302687.608703:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350250240 00000100:00100000:2.0:1789302687.608775:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 522 00000100:00100000:2.0:1789302687.608787:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:23013:x1876219350250240:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302687.614637:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:23013:x1876219350250240:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 5833us (6406us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.614657:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 522 00000100:00100000:1.0:1789302687.618675:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302687.618891:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350250368 00000100:00100000:0.0:1789302687.618904:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350250368 00000100:00100000:0.0:1789302687.618989:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 523 00000100:00100000:0.0:1789302687.619001:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+11:5588:x1876219350250368:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:0.0:1789302687.619026:0:5838:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302687.619052:0:5838:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e0221e000/0x64fe43095184cc9e lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0047 expref: -99 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.619112:0:5838:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e0221e000/0x64fe43095184cc9e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0047 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.619124:0:5838:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e0221e000/0x64fe43095184cc9e lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0047 expref: 12 pid: 5838 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302687.619135:0:5838:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221e000, rc 0) 00000100:00100000:0.0:1789302687.619208:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350250368:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 208us (549us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.619227:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 523 00000100:00100000:3.0:1789302687.624357:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302687.624963:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350250496 00000100:00100000:2.0:1789302687.624977:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350250496 00000100:00100000:2.0:1789302687.625057:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 524 00000100:00100000:2.0:1789302687.625070:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350250496:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302687.625500:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350250496:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 432us (1156us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.625557:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 524 00000100:00100000:3.0:1789302687.627560:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302687.627693:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350250624 00000100:00100000:1.0:1789302687.627705:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350250624 00000100:00100000:1.0:1789302687.627770:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 525 00000100:00100000:1.0:1789302687.627781:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350250624:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 00000100:00100000:1.0:1789302687.628075:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350250624:12345-192.168.203.11@tcp:503:ll_cfg_requeue.0 Request processed in 293us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.628083:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 525 00000100:00100000:1.0:1789302687.631745:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302687.631857:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350250752 00000100:00100000:0.0:1789302687.631871:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350250752 00000100:00100000:0.0:1789302687.631977:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 526 00000100:00100000:0.0:1789302687.631989:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:23014:x1876219350250752:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:0.0:1789302687.632391:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:23014:x1876219350250752:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 402us (660us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302687.632399:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 526 00000100:00100000:3.0:1789302687.634947:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302687.635175:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350250880 00000100:00100000:2.0:1789302687.635188:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350250880 00000100:00100000:2.0:1789302687.635280:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 527 00000100:00100000:2.0:1789302687.635292:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:23014:x1876219350250880:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302687.635651:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:23014:x1876219350250880:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 361us (716us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302687.635660:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 527 00000100:00100000:3.0:1789302687.638126:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302687.638672:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251008 00000100:00100000:1.0:1789302687.638686:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251008 00000100:00100000:1.0:1789302687.638744:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 528 00000100:00100000:1.0:1789302687.638753:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+12:5588:x1876219350251008:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 00010000:00010000:1.0:1789302687.638774:0:5837:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302687.638797:0:5837:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e011eba00/0x64fe43095184cca5 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a004e expref: -99 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.638845:0:5837:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e011eba00/0x64fe43095184cca5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a004e expref: 13 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.638860:0:5837:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e011eba00/0x64fe43095184cca5 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 11 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a004e expref: 13 pid: 5837 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302687.638895:0:5837:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eba00, rc 0) 00000100:00100000:1.0:1789302687.639121:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350251008:12345-192.168.203.11@tcp:101:ll_cfg_requeue.0 Request processed in 368us (1007us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302687.639129:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 528 00000100:00100000:3.0:1789302687.641381:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302687.641537:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251136 00000100:00100000:2.0:1789302687.641549:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251136 00000100:00100000:2.0:1789302687.641704:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 529 00000100:00100000:2.0:1789302687.641715:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350251136:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 00000100:00100000:2.0:1789302687.642084:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:5588:x1876219350251136:12345-192.168.203.11@tcp:501:ll_cfg_requeue.0 Request processed in 368us (711us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302687.642094:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 529 00010000:00010000:0.0:1789302688.396847:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302688.781528:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371092096:0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.781624:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302688.781730:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371092096 00000100:00100000:2.0:1789302688.781740:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371092096 00000100:00100000:2.0:1789302688.781814:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:2.0:1789302688.781835:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371092096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.781918:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371092096:12345-0@lo:400:kworker.0 Request processed in 95us (293us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302688.781926:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00080000:2.0:1789302688.781951:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302688.781991:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371092096:0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.782036:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371092224:0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.782067:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302688.782122:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371092224 00000100:00100000:2.0:1789302688.782126:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371092224 00000100:00100000:2.0:1789302688.782163:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:2.0:1789302688.782171:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371092224:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.782209:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371092224:12345-0@lo:400:kworker.0 Request processed in 39us (141us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302688.782215:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00080000:2.0:1789302688.782231:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302688.782253:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371092224:0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.782315:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371091968:0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.782341:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302688.785136:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371091840:0@lo:400:kworker.0 00000100:00100000:3.0:1789302688.785246:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302688.785332:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371092352:0@lo:400:kworker.0 00000100:00100000:1.0:1789302688.785540:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091840 00000100:00100000:1.0:1789302688.785556:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091840 00000100:00100000:1.0:1789302688.785617:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:1.0:1789302688.785637:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371091840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302688.785727:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371091840:12345-0@lo:400:kworker.0 Request processed in 93us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302688.785734:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1789302688.790702:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371092480:0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.790793:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302688.790883:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371092480 00000100:00100000:2.0:1789302688.791020:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371092480 00000100:00100000:2.0:1789302688.791103:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1789302688.791115:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371092480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.791188:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371092480:12345-0@lo:400:kworker.0 Request processed in 75us (395us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302688.791197:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00080000:2.0:1789302688.791229:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302688.791335:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371092480:0@lo:400:kworker.0 00000100:00100000:1.0:1789302688.791679:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371091968 00000100:00100000:1.0:1789302688.791686:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371091968 00000100:00100000:1.0:1789302688.791730:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1789302688.791739:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371091968:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302688.791801:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371091968:12345-0@lo:400:kworker.0 Request processed in 63us (9458us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302688.791808:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1789302688.791843:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302688.791888:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371091840:0@lo:400:kworker.0 00000100:00100000:3.0:1789302688.792033:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371092352 00000100:00100000:3.0:1789302688.792038:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371092352 00000100:00100000:3.0:1789302688.792096:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 329 00000100:00100000:3.0:1789302688.792104:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371092352:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302688.792144:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371092352:12345-0@lo:400:kworker.0 Request processed in 41us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302688.792150:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 329 00000100:00080000:3.0:1789302688.792178:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302688.792227:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371092352:0@lo:400:kworker.0 00000100:00100000:2.0:1789302688.794167:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371091968:0@lo:400:kworker.0 00000100:00100000:0.0:1789302689.036616:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.036666:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.036701:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.036741:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.036774:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.037035:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.037074:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.037113:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251648 00000100:00100000:0.0:1789302689.037208:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251648 00000100:00100000:0.0:1789302689.037284:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251904 00000100:00100000:0.0:1789302689.037288:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251904 00000100:00100000:0.0:1789302689.037303:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251520 00000100:00100000:0.0:1789302689.037306:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251520 00000100:00100000:0.0:1789302689.037320:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350252032 00000100:00100000:0.0:1789302689.037322:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350252032 00000100:00100000:0.0:1789302689.037340:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:0.0:1789302689.037352:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350251648:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302689.037444:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350251648:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 94us (776us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302689.037452:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:0.0:1789302689.037478:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:0.0:1789302689.037485:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350251904:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302689.037536:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350251904:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 51us (833us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302689.037543:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:0.0:1789302689.037556:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 307 00000100:00100000:0.0:1789302689.037563:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350251520:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302689.037587:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2348:x1876219350251520:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (813us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302689.037594:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 307 00000100:00100000:0.0:1789302689.037607:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 308 00000100:00100000:0.0:1789302689.037613:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350252032:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302689.037639:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350252032:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (564us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302689.037652:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 308 00000100:00100000:1.0:1789302689.038081:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251392 00000100:00100000:1.0:1789302689.038089:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251392 00000100:00100000:1.0:1789302689.038138:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251776 00000100:00100000:1.0:1789302689.038141:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251776 00000100:00100000:1.0:1789302689.038188:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:1.0:1789302689.038196:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2348:x1876219350251392:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302689.038232:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:2348:x1876219350251392:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (1634us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302689.038240:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:1.0:1789302689.038255:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:1.0:1789302689.038262:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2348:x1876219350251776:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302689.038286:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:2348:x1876219350251776:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (1491us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302689.038293:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:3.0:1789302689.038833:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350251264 00000100:00100000:3.0:1789302689.038840:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350251264 00000100:00100000:3.0:1789302689.038886:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 531 00000100:00100000:3.0:1789302689.038893:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350251264:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302689.039039:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350251264:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 146us (2297us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302689.039047:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 531 00000100:00080000:1.0:1789302689.039267:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302689.039274:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302689.039277:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302689.292791:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371092608:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302689.292915:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302689.293073:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371092608 00000100:00100000:0.0:1789302689.293083:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371092608 00000100:00100000:0.0:1789302689.293160:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:0.0:1789302689.293172:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371092608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302689.293219:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302689.293286:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371092608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (371us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302689.293295:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:2.0:1789302689.300728:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371092608:0@lo:13:osp-pre-1-0.0 00000001:02000400:0.0:1789302689.376690:0:17912:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12a: Restore an imported file explicitly ========================================================== 08:31:27 (1789302687) 00010000:00010000:1.0:1789302689.421859:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302689.722272:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.722361:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350252288 00000100:00100000:0.0:1789302689.722375:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350252288 00000100:00100000:0.0:1789302689.722442:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 746 00000100:00100000:0.0:1789302689.722456:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+31:23205:x1876219350252288:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302689.722485:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302689.722546:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccac lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0055 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.722678:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184ccb3 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.722697:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184ccb3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.722712:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184ccb3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302689.722742:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00010000:00010000:0.0:1789302689.723283:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccac lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0055 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.723307:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e600/0x64fe43095184ccb3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0055 expref: 33 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.723318:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302689.723321:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1789302689.723326:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221e600, rc 0) 00000100:00100000:0.0:1789302689.723407:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:23205:x1876219350252288:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 953us (1150us total) trans 4294967787 rc 0/0 00000100:00100000:0.0:1789302689.723416:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 746 00000100:00100000:0.0:1789302689.733252:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302689.733424:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350252416 00000100:00100000:1.0:1789302689.733440:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350252416 00000100:00100000:1.0:1789302689.733620:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 747 00000100:00100000:1.0:1789302689.733633:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:23205:x1876219350252416:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1789302689.734460:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:23205:x1876219350252416:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 823us (1217us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302689.734472:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 747 00000040:00080000:0.0:1789302689.746250:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302689.746261:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302689.746831:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302689.746843:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302689.770710:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302689.771859:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350252544 00000100:00100000:2.0:1789302689.771879:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350252544 00000100:00100000:2.0:1789302689.771961:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 748 00000100:00100000:2.0:1789302689.772052:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:23206:x1876219350252544:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302689.772082:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302689.772111:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184ccc1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a005c expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772200:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772218:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772233:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772250:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772258:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772266:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772557:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772566:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772582:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772588:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302689.772593:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772601:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772615:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:2.0:1789302689.772619:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302689.772625:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772631:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184ccc8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772641:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772648:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772654:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772659:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302689.772662:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772679:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772688:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff960f03927400) 00010000:00010000:2.0:1789302689.772691:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302689.772694:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772700:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cccf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772739:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184ccc1 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a005c expref: 76 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.772751:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:2.0:1789302689.772754:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302689.773903:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184ccc1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a005c expref: 76 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.773926:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfc00, rc 0) 00000100:00100000:2.0:1789302689.774112:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:23206:x1876219350252544:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2135us (3407us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302689.774126:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 748 00000100:00100000:1.0:1789302689.784365:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.784957:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350252800 00000100:00100000:0.0:1789302689.784983:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350252800 00000100:00100000:0.0:1789302689.785057:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 749 00000100:00100000:0.0:1789302689.785069:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:23206:x1876219350252800:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:0.0:1789302689.785147:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302689.785184:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184cb32 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0001 expref: 75 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.785225:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:0.0:1789302689.785232:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302689.785527:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.785545:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.785556:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.785600:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 32 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.785618:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 32 pid: 5845 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302689.785637:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371092736:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302689.785774:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302689.795210:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302689.795402:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbed80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371092736:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302689.795440:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302689.795556:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350252928 00000100:00100000:2.0:1789302689.795566:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350252928 00010000:00010000:2.0:1789302689.795613:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 33 pid: 5845 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302689.795671:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 244 00000100:00100000:2.0:1789302689.795683:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:2347:x1876219350252928:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302689.795703:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302689.795712:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 33 pid: 5845 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.795725:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 33 pid: 5845 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.795753:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184cb5c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a000f expref: 33 pid: 5845 timeout: 626 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.796050:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:2.0:1789302689.796057:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.796069:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.796079:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302689.796146:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302689.796216:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:2347:x1876219350252928:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 535us (1022us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302689.796226:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 244 00010000:00010000:0.0:1789302689.804647:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302689.804667:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.804907:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.804926:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.804949:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302689.806226:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00010000:00010000:1.0:1789302689.806245:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.806255:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.806264:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.806272:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302689.806277:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.806287:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.806307:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927800) 00010000:00010000:1.0:1789302689.806310:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302689.806317:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302689.806325:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184ccd6 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302689.806347:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ea7277f8 reply state 00000000636a9bb2 transno 4294967788 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302689.806372:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302689.806534:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:23206:x1876219350252800:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 21431us (22149us total) trans 4294967788 rc 0/0 00000100:00100000:1.0:1789302689.806545:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 749 00000100:00100000:2.0:1789302689.806936:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371092864:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302689.807028:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302689.808068:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371092864 00000100:00100000:1.0:1789302689.808082:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371092864 00000100:00100000:1.0:1789302689.808142:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 141 00000100:00100000:1.0:1789302689.808152:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371092864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302689.808197:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302689.808281:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371092864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 130us (1252us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302689.808288:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 141 00000100:00100000:3.0:1789302689.808439:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371092864:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302689.812167:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302689.812193:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302689.812202:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302689.812209:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302689.812215:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302689.812225:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302689.812256:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff960f10d24400) 00010000:00010000:3.0:1789302689.812259:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302689.812266:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302689.812274:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184ccdd lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302690.311380:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.311708:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350253056 00000100:00100000:0.0:1789302690.311723:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350253056 00000100:00100000:0.0:1789302690.311821:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 750 00000100:00100000:0.0:1789302690.311834:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:23243:x1876219350253056:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:0.0:1789302690.311866:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302690.311896:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cce4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0063 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.311995:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cceb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.312013:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cceb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.312029:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cceb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302690.312079:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302690.312083:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302690.312105:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cceb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.312137:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cce4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0063 expref: 77 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.312161:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cceb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0063 expref: 76 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.312170:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302690.312193:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302690.312198:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f200, rc 0) 00000100:00100000:0.0:1789302690.312283:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:23243:x1876219350253056:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 452us (917us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302690.312291:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 750 00000100:00100000:1.0:1789302690.320122:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.320208:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350253312 00000100:00100000:2.0:1789302690.320222:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350253312 00000100:00100000:2.0:1789302690.320311:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 751 00000100:00100000:2.0:1789302690.320324:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:23243:x1876219350253312:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:2.0:1789302690.320406:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302690.320441:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cceb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0063 expref: 76 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.320464:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302690.320482:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302690.320753:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:23243:x1876219350253312:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 430us (644us total) trans 0 rc -17/-17 00000100:00100000:2.0:1789302690.320761:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 751 00000100:00100000:1.0:1789302690.327132:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.327407:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350253440 00000100:00100000:0.0:1789302690.327422:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350253440 00000100:00100000:0.0:1789302690.327540:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 752 00000100:00100000:0.0:1789302690.327564:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:23243:x1876219350253440:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:0.0:1789302690.327593:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302690.327625:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184ccf2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a006a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.327711:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184ccf9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.327729:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184ccf9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.327745:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184ccf9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302690.327813:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302690.327827:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302690.327850:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184ccf9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.327876:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184ccf2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a006a expref: 77 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.327923:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184ccf9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a006a expref: 76 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.327932:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302690.327936:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302690.327940:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f400, rc 0) 00000100:00100000:0.0:1789302690.328929:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:23243:x1876219350253440:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 1370us (1803us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302690.328941:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 752 00000100:00100000:0.0:1789302690.350706:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.356243:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350253568 00000100:00100000:2.0:1789302690.356267:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350253568 00000100:00100000:2.0:1789302690.356366:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 753 00000100:00100000:2.0:1789302690.356379:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:23243:x1876219350253568:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:2.0:1789302690.356406:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302690.356433:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd00 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0071 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.356566:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.356585:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.356622:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.356641:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.356648:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.356654:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.357726:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cd15 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.357744:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cd15 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.357755:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cd15 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302690.358078:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000004:00200000:2.0:1789302690.358096:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302690.358127:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cd15 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358145:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358153:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358181:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358188:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.358194:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358204:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358219:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302690.358223:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302690.358229:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358236:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd07 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358255:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358263:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358269:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358275:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.358278:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358296:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358305:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff960f03927400) 00010000:00010000:2.0:1789302690.358308:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302690.358311:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358318:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd0e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358368:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd00 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0071 expref: 78 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358386:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cd15 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0071 expref: 77 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.358395:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927200) 00010000:00010000:2.0:1789302690.358397:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302690.358399:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302690.359704:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23243:x1876219350253568:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 2124us (7810us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.359724:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 753 00010000:00010000:3.0:1789302690.449792:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302690.454138:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.455754:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350253696 00000100:00100000:2.0:1789302690.455775:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350253696 00000100:00100000:2.0:1789302690.455958:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 754 00000100:00100000:2.0:1789302690.455974:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+32:23249:x1876219350253696:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302690.456126:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302690.456156:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd1c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0078 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.456256:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.456274:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.456288:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302690.456335:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302690.456340:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302690.456365:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.456394:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd1c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.456416:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 33 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.456426:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302690.456430:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302690.456435:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe400, rc 0) 00000100:00100000:2.0:1789302690.457164:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:23249:x1876219350253696:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1192us (3038us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.457189:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 754 00000100:00100000:1.0:1789302690.471238:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.477624:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350253952 00000100:00100000:3.0:1789302690.477648:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350253952 00000100:00100000:3.0:1789302690.477804:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 755 00000100:00100000:3.0:1789302690.477826:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23249:x1876219350253952:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302690.477856:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302690.477861:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789302690.477906:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cd15 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0071 expref: 77 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.477930:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927200) 00010000:00010000:3.0:1789302690.477943:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302690.477965:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184cd2a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a007f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.478126:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.478142:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.478178:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.478194:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.478202:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.478209:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479083:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479096:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479105:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302690.479117:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 0 00010000:00010000:3.0:1789302690.479146:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479155:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479162:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479169:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dd800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302690.479182:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479191:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479203:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f0cdf2e00) 00010000:00010000:3.0:1789302690.479206:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302690.479212:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479219:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd31 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302690.479232:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b73a7a5e reply state 00000000ab3a44e0 transno 4294967789 ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302690.479243:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302690.479281:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184cd2a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479300:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 78 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.479308:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302690.479311:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302690.479313:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120ddc00, rc 0) 00000100:00100000:3.0:1789302690.479408:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23249:x1876219350253952:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1593us (8185us total) trans 4294967789 rc 0/0 00000100:00100000:3.0:1789302690.479418:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 755 00000100:00100000:1.0:1789302690.492274:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302690.492324:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.492340:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.492349:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.492356:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302690.492362:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.492372:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.492393:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302690.492397:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302690.492403:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.492410:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd38 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302690.492808:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350254208 00000100:00100000:2.0:1789302690.492834:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350254208 00000100:00100000:2.0:1789302690.492903:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 756 00000100:00100000:2.0:1789302690.492916:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23249:x1876219350254208:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302690.492949:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302690.492975:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd4d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0086 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.493274:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.493292:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.493302:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.493344:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.493360:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 79 pid: 5845 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302690.493376:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371092992:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302690.494202:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33ff9e40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302690.504096:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.504211:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350254336 00000100:00100000:0.0:1789302690.504225:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350254336 00010000:00010000:0.0:1789302690.504283:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 80 pid: 5845 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302690.504288:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371092992:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302690.504341:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 245 00010000:00010000:2.0:1789302690.504343:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302690.504355:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:2347:x1876219350254336:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302690.504377:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302690.504386:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 80 pid: 5845 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.504413:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 80 pid: 5845 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.504441:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cd3f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a007f expref: 80 pid: 5845 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.504480:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302690.504487:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.504542:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.504553:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.504587:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302690.504654:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:2347:x1876219350254336:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 301us (572us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302690.504661:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 245 00010000:00010000:3.0:1789302690.505186:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.505197:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302690.505222:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 0 00010000:00010000:2.0:1789302690.506056:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506068:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506076:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506082:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.506088:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506096:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506113:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:2.0:1789302690.506116:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302690.506122:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506128:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd54 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506210:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd4d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0086 expref: 79 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506222:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302690.506236:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302690.506240:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd4d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0086 expref: 79 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.506251:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfc00, rc 0) 00000100:00100000:2.0:1789302690.506342:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23249:x1876219350254208:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 13429us (14085us total) trans 4294967790 rc 301/301 00000100:00100000:2.0:1789302690.506350:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 756 00000100:00100000:1.0:1789302690.513928:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.514015:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350254592 00000100:00100000:3.0:1789302690.514048:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350254592 00000100:00100000:3.0:1789302690.514154:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 154 00000100:00100000:3.0:1789302690.514166:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23249:x1876219350254592:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302690.514711:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23249:x1876219350254592:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 545us (796us total) trans 4294967791 rc 0/0 00000100:00100000:3.0:1789302690.514720:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 154 00000100:00100000:0.0:1789302690.528169:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302690.528286:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350254720 00000100:00100000:1.0:1789302690.528301:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350254720 00000100:00100000:1.0:1789302690.528430:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 757 00000100:00100000:1.0:1789302690.528442:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23249:x1876219350254720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302690.528477:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302690.528578:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a008d expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302690.528685:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 80 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302690.528702:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 80 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302690.528727:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:1.0:1789302690.528730:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302690.528735:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:1.0:1789302690.528807:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:23249:x1876219350254720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 367us (653us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302690.528826:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 757 00000100:00100000:1.0:1789302690.535428:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.536094:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350254976 00000100:00100000:3.0:1789302690.536109:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350254976 00000100:00100000:3.0:1789302690.536209:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 758 00000100:00100000:3.0:1789302690.536224:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:23249:x1876219350254976:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1789302690.536347:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.536370:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.536384:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.537076:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.537087:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.537152:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.537159:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302690.537165:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.537174:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.537189:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:0.0:1789302690.537192:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1789302690.537199:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.537207:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cd69 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302690.537317:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:23249:x1876219350254976:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 1095us (1901us total) trans 4294967792 rc 0/0 00000100:00100000:0.0:1789302690.537328:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 758 00000100:00100000:0.0:1789302690.547136:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.547793:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350255232 00000100:00100000:3.0:1789302690.547808:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350255232 00000100:00100000:3.0:1789302690.547893:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 759 00000100:00100000:3.0:1789302690.547904:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:23249:x1876219350255232:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302690.548296:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.548318:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.548334:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302690.548428:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967793 00000004:00010000:3.0:1789302690.548473:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000014cdeaee reply state 00000000414b50bc transno 4294967794 ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302690.548484:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302690.550095:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:23249:x1876219350255232:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 2124us (2907us total) trans 4294967794 rc 0/0 00000100:00100000:3.0:1789302690.550106:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 759 00000020:00080000:3.0:1789302690.550114:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789302690.553096:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.553372:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350255488 00000100:00100000:0.0:1789302690.553387:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350255488 00000100:00100000:0.0:1789302690.553469:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 155 00000100:00100000:0.0:1789302690.553481:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ec450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:23249:x1876219350255488:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302690.553853:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ec450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:23249:x1876219350255488:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 372us (771us total) trans 4294967795 rc 0/0 00000100:00100000:0.0:1789302690.553885:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 155 00010000:00010000:2.0:1789302690.556381:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.556473:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.556482:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.556489:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.556495:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.556620:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.556642:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:2.0:1789302690.556645:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789302690.556652:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.556658:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cd70 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302690.611726:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.611932:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350255616 00000100:00100000:2.0:1789302690.611948:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350255616 00000100:00100000:2.0:1789302690.612023:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 760 00000100:00100000:2.0:1789302690.612037:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:23252:x1876219350255616:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302690.612065:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302690.612094:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd77 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0094 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612185:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612205:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612220:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612238:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612245:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612252:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612381:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cd8c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612390:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cd8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612418:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cd8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302690.612455:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000004:00200000:2.0:1789302690.612459:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302690.612479:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cd8c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612494:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612830:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612840:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612847:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.612851:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612860:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612875:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302690.612879:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302690.612885:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612892:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cd7e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612921:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612927:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612933:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612939:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.612941:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612948:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612956:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff960f03927400) 00010000:00010000:2.0:1789302690.612958:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302690.612961:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.612967:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cd85 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.613006:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cd77 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0094 expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.613023:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf800/0x64fe43095184cd8c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0094 expref: 34 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.613044:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:2.0:1789302690.613047:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302690.613050:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf800, rc 0) 00000100:00100000:2.0:1789302690.613261:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23252:x1876219350255616:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1227us (1549us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.613271:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 760 00000100:00100000:1.0:1789302690.620200:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.621729:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350255872 00000100:00100000:3.0:1789302690.621747:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350255872 00000100:00100000:3.0:1789302690.621850:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 761 00000100:00100000:3.0:1789302690.621864:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23252:x1876219350255872:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302690.621901:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302690.621931:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd93 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a009b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622037:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622055:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622069:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622088:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622095:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622102:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622282:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cda8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622292:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cda8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622299:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cda8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302690.622312:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:3.0:1789302690.622432:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622439:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622447:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.622454:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dc200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302690.622459:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624560:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624598:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:3.0:1789302690.624602:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302690.624612:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624648:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cd9a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624671:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624676:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624681:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624685:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302690.624688:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624692:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624698:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302690.624699:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302690.624701:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624704:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184cda1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624777:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184cd93 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a009b expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624799:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cda8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a009b expref: 36 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.624805:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:3.0:1789302690.624806:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302690.624808:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120ddc00, rc 0) 00000100:00100000:3.0:1789302690.625059:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23252:x1876219350255872:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3194us (4872us total) trans 4294967796 rc 0/0 00000100:00100000:3.0:1789302690.625070:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 761 00000100:00100000:3.0:1789302690.628303:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.628413:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350256000 00000100:00100000:0.0:1789302690.628428:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350256000 00000100:00100000:0.0:1789302690.628552:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 762 00000100:00100000:0.0:1789302690.628567:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23252:x1876219350256000:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:0.0:1789302690.628671:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628693:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628708:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628754:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628762:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628770:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628778:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302690.628783:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628792:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628850:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:0.0:1789302690.628854:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302690.628861:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.628869:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdb6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302690.629074:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23252:x1876219350256000:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 509us (784us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302690.629082:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 762 00000100:00100000:1.0:1789302690.630934:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.631049:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350256256 00000100:00100000:3.0:1789302690.631061:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350256256 00000100:00100000:3.0:1789302690.631155:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 156 00000100:00100000:3.0:1789302690.631168:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23252:x1876219350256256:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:2.0:1789302690.631822:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23252:x1876219350256256:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 646us (888us total) trans 4294967797 rc 0/0 00000100:00100000:2.0:1789302690.631833:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 156 00000100:00100000:1.0:1789302690.702220:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.702322:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350256384 00000100:00100000:2.0:1789302690.702338:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350256384 00000100:00100000:2.0:1789302690.702423:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 763 00000100:00100000:2.0:1789302690.702435:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23260:x1876219350256384:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302690.702468:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302690.702496:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdbd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00a2 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.702641:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cdc4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.702659:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cdc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.702674:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cdc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302690.702720:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000004:00200000:2.0:1789302690.702731:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302690.702758:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cdc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.702792:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdbd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00a2 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.702833:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cdc4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00a2 expref: 38 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.702842:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:2.0:1789302690.702846:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302690.702850:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe800, rc 0) 00000100:00100000:2.0:1789302690.702944:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23260:x1876219350256384:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 511us (738us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.702952:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 763 00000100:00100000:3.0:1789302690.706959:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.707013:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350256640 00000100:00100000:2.0:1789302690.707028:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350256640 00000100:00100000:2.0:1789302690.707116:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 31 00000100:00100000:2.0:1789302690.707128:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23260:x1876219350256640:12345-192.168.203.11@tcp:103:lfs.0 00010000:00010000:2.0:1789302690.707148:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302690.707181:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cda8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a009b expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.707207:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:2.0:1789302690.707214:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302690.707290:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23260:x1876219350256640:12345-192.168.203.11@tcp:103:lfs.0 Request processed in 164us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.707321:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 31 00000100:00100000:1.0:1789302690.713586:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.714538:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350256768 00000100:00100000:2.0:1789302690.714555:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350256768 00000100:00100000:2.0:1789302690.714641:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 764 00000100:00100000:2.0:1789302690.714653:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23260:x1876219350256768:12345-192.168.203.11@tcp:58:lfs.0 00010000:00010000:2.0:1789302690.714785:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.714806:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 83 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.714828:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 4/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.714874:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 83 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.716404:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 83 pid: 6919 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302690.716446:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371093120:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302690.716928:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f01254300 going to sleep for 11 seconds 00000100:00100000:1.0:1789302690.721987:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.722081:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350256896 00000100:00100000:0.0:1789302690.722093:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350256896 00000100:00100000:2.0:1789302690.722138:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371093120:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302690.722140:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 84 pid: 6919 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.722176:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302690.722191:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 246 00000100:00100000:0.0:1789302690.722199:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:2347:x1876219350256896:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302690.722215:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302690.722221:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 84 pid: 6919 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.722231:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 84 pid: 6919 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.722253:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184cd62 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a008d expref: 84 pid: 6919 timeout: 627 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.722293:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:0.0:1789302690.722298:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.722307:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.722315:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.722349:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302690.722448:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:2347:x1876219350256896:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 248us (476us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302690.722457:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 246 00010000:00010000:2.0:1789302690.722989:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.723012:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789302690.723325:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].18, 136 off10640 00000100:00100000:2.0:1789302690.723441:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23260:x1876219350256768:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 8787us (9868us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.723453:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 764 00000040:00080000:0.0:1789302690.764828:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302690.764842:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302690.764893:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 19 count 2 00000040:00080000:0.0:1789302690.764904:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302690.765088:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371093248:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302690.765233:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599a80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302690.772645:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.772735:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350257024 00000100:00100000:2.0:1789302690.772749:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350257024 00000100:00100000:2.0:1789302690.772834:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 765 00000100:00100000:2.0:1789302690.772848:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350257024:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302690.772904:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302690.773057:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350257024:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 211us (426us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.773065:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 765 00000100:00100000:0.0:1789302690.773180:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371093248:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302690.777206:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.777827:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350257152 00000100:00100000:0.0:1789302690.777842:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350257152 00000100:00100000:0.0:1789302690.777932:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 766 00000100:00100000:0.0:1789302690.777944:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23270:x1876219350257152:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1789302690.778047:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778087:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778102:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778565:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778590:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778613:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778652:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778661:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778668:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302690.778716:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00000004:00200000:0.0:1789302690.778737:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302690.778753:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778765:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778790:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778796:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778802:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302690.778807:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778823:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778836:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302690.778839:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302690.778844:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778851:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cdd2 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778869:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778875:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778881:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778885:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221f800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302690.778894:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778901:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778909:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x43be2ce7 (ffff960f03501f00) 00010000:00010000:0.0:1789302690.778912:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302690.778915:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778920:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdd9 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x43be2ce7 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778940:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778947:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778953:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778958:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302690.778961:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778966:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778972:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:0.0:1789302690.778974:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302690.778984:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.778989:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cde0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302690.779099:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23270:x1876219350257152:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 1154us (1904us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302690.779108:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 766 00000100:00100000:1.0:1789302690.787217:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.789383:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350257408 00000100:00100000:2.0:1789302690.789400:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350257408 00000100:00100000:2.0:1789302690.789484:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 767 00000100:00100000:2.0:1789302690.789496:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23270:x1876219350257408:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302690.789551:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302690.789573:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cde7 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00a9 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789697:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789711:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789737:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302690.789749:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:2.0:1789302690.789855:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789862:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789868:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789873:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.789877:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789883:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789895:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:2.0:1789302690.789897:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302690.789902:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789907:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184cdee lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789948:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cde7 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00a9 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.789968:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302690.789971:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302690.789975:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cde7 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00a9 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.790299:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbee00, rc 0) 00000100:00100000:2.0:1789302690.790372:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350257408:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 877us (3169us total) trans 4294967798 rc 301/301 00000100:00100000:2.0:1789302690.790379:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 767 00000100:00100000:3.0:1789302690.796760:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.797855:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350257536 00000100:00100000:0.0:1789302690.797867:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350257536 00000100:00100000:0.0:1789302690.797946:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 768 00000100:00100000:0.0:1789302690.797958:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350257536:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302690.797994:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302690.798115:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350257536:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 159us (1362us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302690.798124:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 768 00000100:00100000:3.0:1789302690.807762:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.807906:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350257792 00000100:00100000:0.0:1789302690.807921:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350257792 00000100:00100000:0.0:1789302690.808012:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 769 00000100:00100000:0.0:1789302690.808026:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350257792:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302690.808055:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302690.808059:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789302690.808098:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184cdc4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00a2 expref: 38 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.808124:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:0.0:1789302690.808131:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302690.808164:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdfc lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00b0 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302690.810777:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x10:0x0], index 0: 0 00010000:00010000:0.0:1789302690.811836:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184ce03 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.811863:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184ce03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.811879:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184ce03 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.813703:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cdfc lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b0 expref: 40 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.813733:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184ce03 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b0 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.813744:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f03501f00) 00010000:00010000:0.0:1789302690.813748:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302690.813753:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221fa00, rc 0) 00000100:00100000:0.0:1789302690.813932:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23270:x1876219350257792:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 5908us (6181us total) trans 4294967799 rc 0/0 00000100:00100000:0.0:1789302690.813944:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 769 00000100:00100000:0.0:1789302690.826625:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.832007:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350258048 00000100:00100000:2.0:1789302690.832022:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350258048 00000100:00100000:2.0:1789302690.832087:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 770 00000100:00100000:2.0:1789302690.832097:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23270:x1876219350258048:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302690.832166:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302690.832197:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184ce03 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b0 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832222:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f03501f00) 00010000:00010000:2.0:1789302690.832238:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302690.832264:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832277:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832288:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832324:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832332:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832340:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832410:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832420:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832427:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832432:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.832436:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832444:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832452:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:2.0:1789302690.832456:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302690.832459:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832465:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce11 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832483:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832490:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832497:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832526:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302690.832530:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832535:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832543:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0xa6301165 (ffff960f03927400) 00010000:00010000:2.0:1789302690.832546:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302690.832549:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302690.832554:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce18 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x30:0x0].0xa6301165 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302690.832645:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350258048:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 548us (6034us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302690.832653:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 770 00000100:00100000:3.0:1789302690.848608:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.848650:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350258176 00000100:00100000:3.0:1789302690.848660:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350258176 00000100:00100000:3.0:1789302690.848710:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 771 00000100:00100000:3.0:1789302690.848719:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23270:x1876219350258176:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302690.848738:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302690.848759:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce1f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00b7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.848805:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.848829:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.848840:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.848943:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce1f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.848963:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 39 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.848971:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302690.848974:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302690.848978:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dd200, rc 0) 00000100:00100000:3.0:1789302690.849040:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23270:x1876219350258176:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 316us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302690.849047:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 771 00000100:00100000:3.0:1789302690.857054:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.857092:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350258432 00000100:00100000:3.0:1789302690.857101:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350258432 00000100:00100000:3.0:1789302690.857148:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 772 00000100:00100000:3.0:1789302690.857157:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23270:x1876219350258432:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302690.857235:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.857263:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.857277:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302690.857619:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].8, 88 off8728 00000004:00080000:3.0:1789302690.857630:0:5843:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 527868127054ns, next at 528868125079ns, rc = 0 00000004:00010000:3.0:1789302690.857718:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000084e6d509 reply state 0000000017d620bb transno 4294967800 ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302690.857728:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302690.857824:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23270:x1876219350258432:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 655us (766us total) trans 4294967800 rc 0/0 00000100:00100000:3.0:1789302690.857832:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 772 00000100:00100000:3.0:1789302690.864436:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302690.864535:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350258560 00000100:00100000:2.0:1789302690.864550:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350258560 00000100:00100000:3.0:1789302690.864744:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 773 00000100:00100000:3.0:1789302690.864760:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23270:x1876219350258560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302690.864795:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302690.864833:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00be expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.864891:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.864905:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.864930:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.864942:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.864958:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302690.864968:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.864972:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.864983:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.865002:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302690.865007:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789302690.865012:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.865019:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce2d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.865026:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302690.865032:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302690.865036:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dc200, rc 0) 00000100:00100000:3.0:1789302690.865114:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23270:x1876219350258560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 357us (691us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302690.865122:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 773 00000100:00100000:3.0:1789302690.868943:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302690.870242:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350258688 00000100:00100000:1.0:1789302690.870257:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350258688 00000100:00100000:1.0:1789302690.870335:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 189 00000100:00100000:1.0:1789302690.870345:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350258688:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00000100:00100000:1.0:1789302690.871271:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350258688:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 871us (2289us total) trans 4294967330 rc 0/0 00000100:00100000:1.0:1789302690.871292:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 189 00000100:00100000:2.0:1789302690.882335:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.882891:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350258944 00000100:00100000:3.0:1789302690.882905:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350258944 00000100:00100000:3.0:1789302690.883070:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 157 00000100:00100000:3.0:1789302690.883083:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23270:x1876219350258944:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302690.884287:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23270:x1876219350258944:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1198us (1967us total) trans 4294967801 rc 0/0 00000100:00100000:3.0:1789302690.884299:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 157 00000100:00100000:2.0:1789302690.891740:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.891832:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350259200 00000100:00100000:3.0:1789302690.891845:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350259200 00000100:00100000:3.0:1789302690.892038:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 774 00000100:00100000:3.0:1789302690.892052:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23270:x1876219350259200:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302690.892212:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23270:x1876219350259200:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 163us (489us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302690.892219:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 774 00000100:00100000:2.0:1789302690.904180:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.904660:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350259328 00000100:00100000:3.0:1789302690.904673:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350259328 00000100:00100000:3.0:1789302690.904751:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 775 00000100:00100000:3.0:1789302690.904777:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23270:x1876219350259328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302690.904804:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302690.904843:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce3b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00c5 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.908710:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce42 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.908747:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce42 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.908765:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce42 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302690.908845:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302690.908877:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce42 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.908932:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce3b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00c5 expref: 44 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.908957:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce42 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00c5 expref: 43 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.909026:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302690.909030:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1789302690.909035:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dda00, rc 0) 00000100:00100000:3.0:1789302690.909149:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23270:x1876219350259328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4372us (4982us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302690.909177:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 775 00000100:00100000:1.0:1789302690.932828:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302690.932867:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350259456 00000100:00100000:1.0:1789302690.932876:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350259456 00000100:00100000:1.0:1789302690.932928:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 190 00000100:00100000:1.0:1789302690.932937:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350259456:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302690.932954:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302690.933222:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00cc expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302690.933251:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x10:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6aa697a2, blocks: 0x1 00010000:00010000:1.0:1789302690.935135:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=527s v=(0 0) 00010000:00010000:1.0:1789302690.935144:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302690.935158:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302690.935184:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302690.935198:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea800, rc 0) 00000100:00100000:1.0:1789302690.935283:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350259456:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2345us (2472us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302690.935292:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 190 00000100:00100000:3.0:1789302690.939706:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.939745:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350259712 00000100:00100000:3.0:1789302690.939754:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350259712 00000100:00100000:3.0:1789302690.939809:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 776 00000100:00100000:3.0:1789302690.939829:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23270:x1876219350259712:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302690.939934:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23270:x1876219350259712:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 106us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302690.939941:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 776 00000100:00100000:2.0:1789302690.944156:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302690.944295:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350259840 00000100:00100000:0.0:1789302690.944311:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350259840 00000100:00100000:0.0:1789302690.944633:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 191 00000100:00100000:0.0:1789302690.944645:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:23270:x1876219350259840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302690.944673:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302690.944692:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00d3 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.944726:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.944753:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302690.944758:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302690.944763:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.944800:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302690.944826:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 11 pid: 6696 timeout: 627 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302690.944841:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371093376:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302690.944958:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302690.955557:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.955636:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350259968 00000100:00100000:3.0:1789302690.955656:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350259968 00010000:00010000:3.0:1789302690.955724:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 12 pid: 6696 timeout: 627 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302690.955767:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371093376:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302690.955788:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 247 00000100:00100000:3.0:1789302690.955803:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b3450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350259968:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302690.955820:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.955841:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302690.955845:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221fe00, rc 0) 00010000:00010000:3.0:1789302690.955876:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 12 pid: 6696 timeout: 627 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.955896:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 12 pid: 6696 timeout: 627 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302690.955907:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:23270:x1876219350259840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 11260us (11758us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302690.955920:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 191 00010000:00010000:3.0:1789302690.955940:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184ce49 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00cc expref: 12 pid: 6696 timeout: 627 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.955975:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302690.955980:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302690.955992:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.956005:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302690.956065:0:8463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302690.956084:0:8463:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8463:1876219371093504:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302690.956276:0:8463:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6900 going to sleep for 11 seconds 00000100:00100000:3.0:1789302690.959568:0:8463:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb9180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8463:1876219371093504:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302690.959601:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302690.959664:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b3450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350259968:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 3864us (4115us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302690.959678:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 247 00000100:00100000:3.0:1789302690.962692:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302690.962830:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350260096 00000100:00100000:3.0:1789302690.962844:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350260096 00002000:00100000:3.0:1789302690.962916:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x10:0x0] (0->157) req@ffff960f25fb9180 x1876219350260096/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:266/0 lens 648/0 e 0 to 0 dl 1789302701 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302690.962929:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fb9180 with x1876219350260096 ext(0->157) 00002000:00010000:3.0:1789302690.962956:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f25fb9180 00000100:00100000:3.0:1789302690.962992:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 63 00000100:00100000:3.0:1789302690.963003:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350260096:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1789302690.963609:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350260096:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 604us (929us total) trans 4294967331 rc 0/0 00000100:00100000:3.0:1789302690.963620:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 63 00002000:00100000:3.0:1789302690.963628:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x10:0x0] (0->157) req@ffff960f25fb9180 x1876219350260096/t4294967331(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:266/0 lens 648/448 e 0 to 0 dl 1789302701 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302690.963638:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fb9180 with x1876219350260096 ext(0->157) 00002000:00010000:3.0:1789302690.963642:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f25fb9180 00000100:00100000:2.0:1789302690.988010:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302690.988102:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350260224 00000100:00100000:1.0:1789302690.988116:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350260224 00000100:00100000:1.0:1789302690.988205:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 777 00000100:00100000:1.0:1789302690.988217:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23270:x1876219350260224:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:2.0:1789302691.115899:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967799 is committed 00000001:00080000:2.0:1789302691.115922:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967795 is committed 00000001:00080000:3.0:1789302691.116064:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967801 is committed 00000004:00080000:3.0:1789302691.116077:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 9 committed 00000100:00100000:0.0:1789302691.116643:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371093632:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1789302691.116935:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23270:x1876219350260224:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 128714us (128936us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302691.116949:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 777 00000100:00100000:0.0:1789302691.117713:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302691.122921:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.123296:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371093632 00000100:00100000:3.0:1789302691.123310:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371093632 00000100:00080000:0.0:1789302691.123381:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302691.123399:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350260352 00000100:00100000:3.0:1789302691.123404:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350260352 00000100:00100000:3.0:1789302691.123447:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:3.0:1789302691.123459:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371093632:12345-0@lo:2:osp-syn-0-0.0 00000100:00080000:0.0:1789302691.123477:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:3.0:1789302691.123732:0:6696:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1789302691.123736:0:6696:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb mtime from disk: 0 -> 1789302690 00002000:00010000:3.0:1789302691.123739:0:6696:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1789302691.123742:0:6696:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb ctime from disk: 1789302690 -> 1789302690 00002000:00010000:3.0:1789302691.123744:0:6696:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1789302691.123842:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371093632:12345-0@lo:2:osp-syn-0-0.0 Request processed in 385us (6130us total) trans 4294967332 rc 0/0 00000004:00080000:0.0:1789302691.123919:0:3300:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3a9e0380/2, rc 0, transno 4294967332 00000100:00100000:0.0:1789302691.123928:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371093632:0@lo:2:osp-syn-0-0.0 00000100:00100000:1.0:1789302691.132604:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 193 00000100:00100000:1.0:1789302691.133411:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350260352:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1789302691.135413:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000001:00080000:0.0:1789302691.195778:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967330 is committed 00000001:00080000:0.0:1789302691.195798:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967332 is committed 00000001:00080000:0.0:1789302691.195804:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967331 is committed 00000100:00100000:1.0:1789302691.196058:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350260352:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 62669us (73145us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302691.196072:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 193 00000100:00100000:1.0:1789302691.201355:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302691.203300:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350260608 00000100:00100000:2.0:1789302691.203318:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350260608 00000100:00080000:0.0:1789302691.203382:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302691.203450:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 778 00000100:00100000:2.0:1789302691.203532:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23270:x1876219350260608:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302691.203602:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302691.203641:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ce42 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00c5 expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.203660:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:2.0:1789302691.203678:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302691.203719:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.203745:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.203758:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302691.205852:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002a2c4763 reply state 0000000068ce6ceb transno 4294967802 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302691.205871:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302691.206344:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23270:x1876219350260608:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 2837us (4992us total) trans 4294967802 rc 0/0 00000100:00100000:2.0:1789302691.206355:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 778 00000100:00100000:1.0:1789302691.214637:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302691.216052:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350260736 00000100:00100000:2.0:1789302691.216070:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350260736 00000100:00100000:2.0:1789302691.216171:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 194 00000100:00100000:2.0:1789302691.216184:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350260736:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1789302691.216553:0:8571:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:2.0:1789302691.216558:0:8571:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb mtime from disk: 1789302690 -> 1536580263 00002000:00010000:2.0:1789302691.216562:0:8571:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb atime from disk: 0 -> 1789302187 00002000:00010000:2.0:1789302691.216564:0:8571:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb ctime from disk: 1789302690 -> 1789302690 00002000:00010000:2.0:1789302691.216566:0:8571:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x10:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1789302691.216667:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350260736:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 485us (2045us total) trans 4294967333 rc 0/0 00000100:00100000:2.0:1789302691.216677:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 194 00010000:00010000:3.0:1789302691.219929:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.219952:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.219972:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.219981:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.219987:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.219996:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.220018:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302691.220022:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302691.220028:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.220035:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184ce57 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302691.223365:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.223457:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350260992 00000100:00100000:3.0:1789302691.223471:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350260992 00000100:00100000:3.0:1789302691.223568:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 779 00000100:00100000:3.0:1789302691.223581:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23270:x1876219350260992:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1789302691.223797:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.223827:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 36 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.223853:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 4/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.223896:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 36 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.223912:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 36 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.223928:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371093760:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302691.224051:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6e40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302691.234659:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.234700:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbb480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371093760:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302691.234717:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350261120 00000100:00100000:2.0:1789302691.234737:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350261120 00010000:00010000:3.0:1789302691.234747:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.234786:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302691.234847:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 248 00000100:00100000:2.0:1789302691.234859:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2347:x1876219350261120:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302691.234880:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302691.234888:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.234903:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.234930:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce26 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x14:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00b7 expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.234946:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:2.0:1789302691.234951:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.234962:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.234971:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.235004:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302691.235068:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2347:x1876219350261120:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 210us (427us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302691.235086:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 248 00010000:00010000:3.0:1789302691.235423:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235434:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235482:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235490:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235497:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235534:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.235539:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235548:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235559:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302691.235562:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1789302691.235566:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.235572:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce5e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.235635:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23270:x1876219350260992:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 12056us (12284us total) trans 0 rc -22/-22 00000100:00100000:3.0:1789302691.235644:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 779 00000100:00100000:3.0:1789302691.239458:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302691.239636:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350261248 00000100:00100000:0.0:1789302691.239650:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350261248 00000100:00100000:0.0:1789302691.239736:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 195 00000100:00100000:0.0:1789302691.239750:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350261248:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302691.239819:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.239847:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.239859:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x10:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789302691.239864:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302691.239870:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.239916:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.239933:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 10 pid: 8571 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302691.239950:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371093888:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302691.240067:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e5996c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302691.241966:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371093888:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302691.242006:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.244285:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302691.244750:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350261376 00000100:00100000:1.0:1789302691.244763:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350261376 00010000:00010000:1.0:1789302691.244826:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 11 pid: 8571 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302691.244873:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:1.0:1789302691.244890:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350261376:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302691.244909:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302691.244936:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 11 pid: 8571 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.244952:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 11 pid: 8571 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.245004:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001389b032/0x64fe43095184ce50 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00d3 expref: 11 pid: 8571 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.245022:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302691.245028:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302691.245035:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.245049:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.245096:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302691.245181:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350261376:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 299us (908us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302691.245189:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 249 00010000:00010000:3.0:1789302691.245805:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245827:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245865:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245901:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245910:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245917:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221ee00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302691.245922:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245932:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245948:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.245955:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007581eecb/0x64fe43095184ce65 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.246069:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350261248:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 6321us (6625us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302691.246078:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 195 00000100:00100000:3.0:1789302691.252549:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302691.252713:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350261632 00000100:00100000:1.0:1789302691.252727:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350261632 00000100:00100000:1.0:1789302691.252806:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 780 00000100:00100000:1.0:1789302691.252827:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23270:x1876219350261632:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1789302691.252947:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.252969:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.252984:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.253311:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.253325:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 36 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.253335:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.253376:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 36 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302691.253393:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 36 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302691.253410:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371094016:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302691.257339:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302691.270721:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302691.271666:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371094016:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302691.271712:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.271866:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350261760 00000100:00100000:3.0:1789302691.271875:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350261760 00010000:00010000:3.0:1789302691.271936:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.271979:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 250 00000100:00100000:3.0:1789302691.271991:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2348:x1876219350261760:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302691.272009:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302691.272017:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.272032:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.272059:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce34 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00be expref: 37 pid: 5845 timeout: 628 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.272075:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302691.272080:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.272092:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.272101:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.272129:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302691.272196:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350261760:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 207us (1487us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302691.272204:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 250 00010000:00010000:3.0:1789302691.272335:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.272343:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302691.272672:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967803 00000001:00080000:3.0:1789302691.273005:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967804 00000001:00080000:3.0:1789302691.273059:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967805 00000100:00100000:3.0:1789302691.273161:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371094144:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789302691.273245:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302691.273281:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6180 going to sleep for 11 seconds 00000100:00100000:0.0:1789302691.273623:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371094144 00000100:00100000:0.0:1789302691.273633:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371094144 00000100:00100000:0.0:1789302691.273682:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 25 00000100:00100000:0.0:1789302691.273701:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371094144:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302691.273796:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371094144:12345-0@lo:1000:mdt00_001.0 Request processed in 106us (551us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302691.273804:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 25 00000100:00100000:3.0:1789302691.274078:0:5844:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967332 gen 2 00000004:00080000:3.0:1789302691.274086:0:5844:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3a9e0380, transno 4294967332 00000100:00100000:3.0:1789302691.274109:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb9500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371094144:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1789302691.278483:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967806 00000100:00100000:1.0:1789302691.278677:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371094272:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1789302691.278779:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302691.278828:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302691.279051:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371094272 00000100:00100000:0.0:1789302691.279061:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371094272 00000100:00100000:0.0:1789302691.279112:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 26 00000100:00100000:0.0:1789302691.279122:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371094272:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302691.279392:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5844:x1876219371094272:12345-0@lo:1000:mdt00_001.0 Request processed in 271us (613us total) trans 4294967334 rc 0/0 00000100:00100000:0.0:1789302691.279399:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 26 00000100:00100000:3.0:1789302691.279540:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371094272:0@lo:1000:mdt00_001.0 00010000:00010000:3.0:1789302691.279587:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279598:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279606:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279612:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.279617:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279626:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279661:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302691.279664:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302691.279671:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279678:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184ce73 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x14:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279841:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279849:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279855:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279861:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.279864:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279871:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279880:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:3.0:1789302691.279882:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302691.279885:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 2/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279892:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cdcb lrc: 0/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279930:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 3/0,1 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279944:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279957:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279965:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279971:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279977:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279982:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dd800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.279985:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279991:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.279999:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:3.0:1789302691.280001:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302691.280004:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 2/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280010:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce7a lrc: 0/0,0 mode: --/EX res: [0x200000401:0x32:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280028:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280035:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280041:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280047:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.280050:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280056:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280063:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f10d24300) 00010000:00010000:3.0:1789302691.280065:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302691.280068:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.280073:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184ce6c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x32:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.280422:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23270:x1876219350261632:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 27593us (27917us total) trans 4294967807 rc 0/0 00000100:00100000:3.0:1789302691.280432:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 780 00000020:00080000:3.0:1789302691.280437:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:0.0:1789302691.283494:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302691.285618:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302691.286131:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:0.0:1789302691.286142:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302691.294704:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302691.294893:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350262016 00000100:00100000:2.0:1789302691.294907:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350262016 00000100:00100000:2.0:1789302691.294975:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 158 00000100:00100000:2.0:1789302691.294987:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23270:x1876219350262016:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1789302691.296094:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x14:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1789302691.296216:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967808 00010000:00010000:2.0:1789302691.296282:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.296303:0:12976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.296318:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.296328:0:12976:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.296335:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.296342:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.296367:0:12976:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789302691.297928:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302691.297941:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.297952:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302691.297957:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.297978:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.297997:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x14:0x0].0x0 (ffff960f03927400) 00010000:00010000:0.0:1789302691.298001:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302691.298007:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.298013:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184ce81 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x14:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.298034:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302691.298036:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302691.298037:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302691.300788:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23270:x1876219350262016:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1513us (1810us total) trans 4294967809 rc 0/0 00000100:00100000:2.0:1789302691.300830:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 158 00000020:00080000:2.0:1789302691.300854:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1789302691.477260:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302691.735951:0:17963:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302691.735968:0:17963:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302691.736010:0:17963:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 19 count 2 00000040:00080000:3.0:1789302691.736016:0:17963:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302691.859454:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.859524:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350262272 00000100:00100000:3.0:1789302691.859536:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350262272 00000100:00100000:3.0:1789302691.859601:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 781 00000100:00100000:3.0:1789302691.859614:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23273:x1876219350262272:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302691.859649:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302691.859675:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce88 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00da expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.859771:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.859788:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.859801:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.859825:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.859832:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.859838:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860258:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce9d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860274:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860283:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce9d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302691.860296:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:3.0:1789302691.860415:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860423:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860432:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860440:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dc200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.860445:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860455:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860470:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:3.0:1789302691.860474:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302691.860479:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860487:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184ce8f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860527:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860543:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860549:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860554:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.860557:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860563:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860585:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302691.860588:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302691.860591:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860597:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184ce96 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860652:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ce88 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00da expref: 44 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860667:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd200/0x64fe43095184ce9d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00da expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.860675:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2e00) 00010000:00010000:3.0:1789302691.860677:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302691.860686:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dd200, rc 0) 00000100:00100000:3.0:1789302691.860778:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23273:x1876219350262272:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1166us (1335us total) trans 4294967810 rc 0/0 00000100:00100000:3.0:1789302691.860787:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 781 00000100:00100000:0.0:1789302691.871774:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.872191:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350262400 00000100:00100000:3.0:1789302691.872204:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350262400 00000100:00100000:3.0:1789302691.872280:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 782 00000100:00100000:3.0:1789302691.872292:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23273:x1876219350262400:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:3.0:1789302691.872392:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872413:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872427:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872470:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872479:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872485:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872492:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.872497:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872542:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872578:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2e00) 00010000:00010000:3.0:1789302691.872581:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302691.872588:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 2/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.872595:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceab lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302691.872691:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23273:x1876219350262400:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 400us (931us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302691.872699:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 782 00000100:00100000:0.0:1789302691.883368:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.885719:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350262656 00000100:00100000:3.0:1789302691.885742:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350262656 00000100:00100000:3.0:1789302691.885869:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 159 00000100:00100000:3.0:1789302691.885896:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23273:x1876219350262656:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:3.0:1789302691.886824:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:23273:x1876219350262656:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 927us (3451us total) trans 4294967811 rc 0/0 00000100:00100000:3.0:1789302691.886834:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 159 00000100:00100000:0.0:1789302691.976422:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.976537:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350262784 00000100:00100000:3.0:1789302691.976551:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350262784 00000100:00100000:3.0:1789302691.976637:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 783 00000100:00100000:3.0:1789302691.976649:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+75:23289:x1876219350262784:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302691.976694:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302691.976723:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceb2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00e1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.976828:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.976857:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.976873:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.976891:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.976899:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.976907:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977273:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cec7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977283:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cec7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977291:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cec7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302691.977323:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x30:0x0]: returning size 16384 00000004:00200000:3.0:1789302691.977327:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302691.977349:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cec7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977363:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977382:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977389:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977396:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dd800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.977401:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977409:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977422:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:3.0:1789302691.977425:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302691.977431:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977438:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184ceb9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977453:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977460:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977465:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977470:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302691.977479:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977485:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977493:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x5b990ec8 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302691.977496:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302691.977499:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977534:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184cec0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x5b990ec8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977572:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184ceb2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00e1 expref: 77 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977589:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc200/0x64fe43095184cec7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00e1 expref: 76 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.977597:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:3.0:1789302691.977600:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789302691.977603:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dc200, rc 0) 00000100:00100000:3.0:1789302691.977681:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:23289:x1876219350262784:12345-192.168.203.11@tcp:101:diff.0 Request processed in 1034us (1276us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302691.977689:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 783 00000100:00080000:0.0:1789302691.978099:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302691.982594:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302691.983224:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350262912 00000100:00100000:3.0:1789302691.983239:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350262912 00000100:00100000:3.0:1789302691.983464:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 784 00000100:00100000:3.0:1789302691.983478:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+76:23289:x1876219350262912:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302691.983534:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302691.983562:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184cece lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00e8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.983651:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.983670:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.983683:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.983699:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.983706:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302691.983712:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.984885:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cee3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.984900:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.984912:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302691.984964:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00000004:00200000:0.0:1789302691.984970:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302691.984995:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985024:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cee3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985034:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985040:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985047:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985053:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302691.985058:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985067:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985095:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x0 (ffff960f03927300) 00010000:00010000:0.0:1789302691.985098:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302691.985104:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985111:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184ced5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985123:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985129:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985143:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985148:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302691.985151:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985158:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985167:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x30:0x0].0x3b61a3c9 (ffff960f0cdf2b00) 00010000:00010000:0.0:1789302691.985169:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302691.985172:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985178:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184cedc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x30:0x0].0x3b61a3c9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985223:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184cece lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00e8 expref: 78 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985239:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f800/0x64fe43095184cee3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00e8 expref: 77 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302691.985246:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2e00) 00010000:00010000:0.0:1789302691.985249:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302691.985251:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f800, rc 0) 00000100:00100000:0.0:1789302691.985340:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23289:x1876219350262912:12345-192.168.203.11@tcp:101:diff.0 Request processed in 1864us (2761us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302691.985358:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 784 00000100:00100000:0.0:1789302691.989150:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302691.989314:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350263040 00000100:00100000:2.0:1789302691.989328:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350263040 00000100:00100000:2.0:1789302691.989417:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 196 00000100:00100000:2.0:1789302691.989429:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+9:2347:x1876219350263040:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:2.0:1789302691.989452:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302691.989480:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000056bdb68c/0x64fe43095184ceea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00ef expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302691.989542:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x10:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6aa695ab, ctime: 0x6aa697a2, blocks: 0xb 00010000:00010000:2.0:1789302691.989569:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=528s v=(0 0) 00010000:00010000:2.0:1789302691.989574:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302691.989585:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000056bdb68c/0x64fe43095184ceea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00ef expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.989605:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000056bdb68c/0x64fe43095184ceea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00ef expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302691.989619:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe200, rc 0) 00000100:00100000:2.0:1789302691.989689:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350263040:12345-192.168.203.11@tcp:101:diff.0 Request processed in 263us (554us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302691.989696:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 196 00000100:00080000:0.0:1789302691.989982:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302692.002945:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302692.005475:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350263296 00000100:00100000:3.0:1789302692.005494:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350263296 00000100:00100000:3.0:1789302692.005583:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 785 00000100:00100000:3.0:1789302692.005595:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23289:x1876219350263296:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302692.005623:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302692.005650:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cef1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00f6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302692.005815:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cef8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302692.005834:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cef8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302692.005854:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cef8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302692.005870:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x32:0x0]: returning size 158 00010000:00010000:3.0:1789302692.006039:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cef1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x30:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00f6 expref: 80 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302692.006059:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cef8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00f6 expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302692.006070:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2e00) 00010000:00010000:3.0:1789302692.006074:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302692.006078:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:3.0:1789302692.006165:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23289:x1876219350263296:12345-192.168.203.11@tcp:101:diff.0 Request processed in 572us (3232us total) trans 4294967812 rc 0/0 00000100:00100000:3.0:1789302692.006173:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 785 00000100:00100000:2.0:1789302692.020613:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302692.020656:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302692.020688:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350263424 00000100:00100000:2.0:1789302692.020699:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350263424 00002000:00100000:2.0:1789302692.020769:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x10:0x0] (0->4095) req@ffff960f3a9e0380 x1876219350263424/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:268/0 lens 488/0 e 0 to 0 dl 1789302703 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302692.020782:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3a9e0380 with x1876219350263424 ext(0->4095) 00002000:00010000:2.0:1789302692.020790:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3a9e0380 00000100:00100000:2.0:1789302692.020834:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 64 00000100:00100000:2.0:1789302692.020845:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350263424:12345-192.168.203.11@tcp:3:diff.0 00000100:00100000:3.0:1789302692.021927:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350263552 00000100:00100000:3.0:1789302692.021937:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350263552 00000100:00100000:3.0:1789302692.021994:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 32 00000100:00100000:3.0:1789302692.022004:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:2347:x1876219350263552:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302692.022019:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302692.022061:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cef8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00f6 expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302692.022199:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x32:0x0].0x0 (ffff960f0cdf2e00) 00010000:00010000:3.0:1789302692.022206:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302692.022285:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:2347:x1876219350263552:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 282us (1627us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302692.022293:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 32 00000100:00100000:2.0:1789302692.022816:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350263424:12345-192.168.203.11@tcp:3:diff.0 Request processed in 1958us (2202us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1789302692.022826:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 64 00002000:00100000:2.0:1789302692.022837:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x10:0x0] (0->4095) req@ffff960f3a9e0380 x1876219350263424/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:268/0 lens 488/4536 e 0 to 0 dl 1789302703 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302692.022847:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3a9e0380 with x1876219350263424 ext(0->4095) 00002000:00010000:2.0:1789302692.022852:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3a9e0380 00000100:00100000:1.0:1789302692.034479:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302692.034546:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350263808 00000100:00100000:1.0:1789302692.034580:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350263808 00000100:00100000:1.0:1789302692.034644:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 160 00000100:00100000:1.0:1789302692.034656:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23289:x1876219350263808:12345-192.168.203.11@tcp:35:diff.0 00000001:00080000:1.0:1789302692.035056:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967813 00000100:00100000:1.0:1789302692.035188:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:23289:x1876219350263808:12345-192.168.203.11@tcp:35:diff.0 Request processed in 532us (729us total) trans 4294967814 rc 0/0 00000100:00100000:1.0:1789302692.035197:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 160 00000020:00080000:1.0:1789302692.035202:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1789302692.492797:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302693.517184:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302693.901692:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302693.901724:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371094400:0@lo:400:kworker.0 00000100:00100000:1.0:1789302693.901742:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371094528:0@lo:400:kworker.0 00000100:00100000:2.0:1789302693.901788:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371094912:0@lo:400:kworker.0 00000100:00100000:0.0:1789302693.902061:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302693.902126:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371094784:0@lo:400:kworker.0 00000100:00100000:0.0:1789302693.902156:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302693.902197:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371094784 00000100:00100000:3.0:1789302693.902199:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371094400 00000100:00100000:3.0:1789302693.902207:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371094400 00000100:00100000:0.0:1789302693.902207:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371094784 00000100:00100000:3.0:1789302693.902261:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 532 00000100:00100000:0.0:1789302693.902261:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 309 00000100:00100000:3.0:1789302693.902270:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371094400:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302693.902270:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371094784:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302693.902324:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371094784:12345-0@lo:400:kworker.0 Request processed in 56us (167us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302693.902331:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 309 00000100:00080000:0.0:1789302693.902362:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302693.902395:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371094784:0@lo:400:kworker.0 00000100:00100000:0.0:1789302693.902406:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371094400:0@lo:400:kworker.0 00000100:00100000:3.0:1789302693.902435:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371094400:12345-0@lo:400:kworker.0 Request processed in 57us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302693.902441:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 532 00000100:00100000:2.0:1789302693.902569:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302693.902728:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302693.902755:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371095040:0@lo:400:kworker.0 00000100:00100000:1.0:1789302693.902777:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302693.902806:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302693.902812:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371094656:0@lo:400:kworker.0 00000100:00100000:1.0:1789302693.902834:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302693.902907:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371094912 00000100:00100000:3.0:1789302693.902913:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371094912 00000100:00100000:3.0:1789302693.902956:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371094528 00000100:00100000:3.0:1789302693.902959:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371094528 00000100:00100000:3.0:1789302693.902970:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371094656 00000100:00100000:3.0:1789302693.902972:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371094656 00000100:00100000:2.0:1789302693.903195:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1789302693.903205:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371094912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302693.903251:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371094912:12345-0@lo:400:kworker.0 Request processed in 48us (681us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302693.903258:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:2.0:1789302693.903277:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 333 00000100:00100000:2.0:1789302693.903284:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371094528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302693.903317:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371094528:12345-0@lo:400:kworker.0 Request processed in 33us (588us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302693.903405:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1789302693.903414:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371094656:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302693.903447:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371094656:12345-0@lo:400:kworker.0 Request processed in 36us (612us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302693.903453:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1789302693.903476:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095040 00000100:00100000:3.0:1789302693.903487:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095040 00000100:00100000:3.0:1789302693.903545:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:3.0:1789302693.903551:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371095040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302693.903585:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371095040:12345-0@lo:400:kworker.0 Request processed in 34us (806us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302693.903591:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00080000:1.0:1789302693.903758:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302693.903763:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302693.903827:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 333 00000100:00100000:2.0:1789302693.903874:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371094912:0@lo:400:kworker.0 00000100:00100000:2.0:1789302693.903909:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371094656:0@lo:400:kworker.0 00000100:00100000:2.0:1789302693.903921:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371094528:0@lo:400:kworker.0 00000100:00100000:2.0:1789302693.903932:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371095040:0@lo:400:kworker.0 00000100:00100000:3.0:1789302694.159306:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302694.159367:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302694.159403:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302694.159425:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302694.159443:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302694.159462:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302694.159485:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302694.159902:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350263936 00000100:00100000:0.0:1789302694.159915:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350263936 00000100:00100000:3.0:1789302694.159920:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264192 00000100:00100000:3.0:1789302694.159924:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264192 00000100:00100000:3.0:1789302694.159988:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264704 00000100:00100000:3.0:1789302694.159992:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264704 00000100:00100000:3.0:1789302694.160023:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264320 00000100:00100000:3.0:1789302694.160026:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264320 00000100:00100000:3.0:1789302694.160040:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264576 00000100:00100000:3.0:1789302694.160043:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264576 00000100:00100000:3.0:1789302694.160077:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00100000:0.0:1789302694.160086:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00100000:3.0:1789302694.160088:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350264192:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302694.160092:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350263936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302694.160176:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350263936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 82us (884us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302694.160197:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00080000:1.0:1789302694.160199:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302694.160204:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302694.160246:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 312 00000100:00100000:0.0:1789302694.160253:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350264704:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302694.160358:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350264192:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 271us (990us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302694.160364:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00100000:3.0:1789302694.160379:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 313 00000100:00100000:3.0:1789302694.160385:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350264320:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302694.160416:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350264320:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (953us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302694.160422:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 313 00000100:00100000:3.0:1789302694.160434:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 314 00000100:00100000:3.0:1789302694.160440:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350264576:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302694.160460:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350264576:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (974us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302694.160465:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 314 00000100:00100000:3.0:1789302694.160485:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264448 00000100:00100000:3.0:1789302694.160489:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264448 00000100:00100000:3.0:1789302694.160552:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264064 00000100:00100000:3.0:1789302694.160555:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264064 00000100:00100000:3.0:1789302694.160578:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:3.0:1789302694.160585:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2348:x1876219350264448:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302694.160622:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2348:x1876219350264448:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (1219us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302694.160629:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:3.0:1789302694.160641:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 336 00000100:00100000:3.0:1789302694.160647:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:2348:x1876219350264064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302694.160672:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:2348:x1876219350264064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (1229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302694.160678:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 336 00000100:00100000:0.0:1789302694.167572:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350264704:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 7301us (8126us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302694.167591:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 312 00000100:00080000:0.0:1789302694.168907:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302694.413044:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371095168:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302694.413145:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302694.413344:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095168 00000100:00100000:3.0:1789302694.413355:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095168 00000100:00100000:3.0:1789302694.413456:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:3.0:1789302694.413467:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371095168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302694.413557:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302694.421944:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371095168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 8463us (8783us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302694.421962:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:1.0:1789302694.422595:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371095168:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302694.541659:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302694.924887:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371095296:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302694.924984:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302694.927159:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095296 00000100:00100000:2.0:1789302694.927179:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095296 00000100:00100000:3.0:1789302694.927379:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 143 00000100:00100000:3.0:1789302694.927394:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371095296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302694.927448:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302694.927770:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371095296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 378us (2784us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302694.927781:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 143 00000100:00100000:2.0:1789302694.930972:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371095296:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302695.565388:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302696.559380:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967814 is committed 00000001:00080000:2.0:1789302696.559398:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967803 is committed 00000001:00080000:2.0:1789302696.559405:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967809 is committed 00000001:00080000:2.0:1789302696.559423:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967811 is committed 00000020:00080000:2.0:1789302696.559459:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f3efb66c0 refcount 2 committed 0 result 0 batchid 11 00000020:00080000:2.0:1789302696.559466:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f12140700 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f3fe80900 00000020:00080000:2.0:1789302696.559470:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f12140000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f25e8e800 00000020:00080000:2.0:1789302696.559525:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 11 committed batchid 10 00000020:00080000:2.0:1789302696.559750:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 10 00000001:00080000:0.0:1789302696.567418:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967334 is committed 00000001:00080000:0.0:1789302696.567442:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967333 is committed 00010000:00010000:0.0:1789302696.589419:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302697.601843:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302697.602236:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264832 00000100:00100000:3.0:1789302697.602253:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264832 00000100:00100000:3.0:1789302697.602417:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 786 00000100:00100000:3.0:1789302697.602460:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23205:x1876219350264832:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1789302697.602647:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23205:x1876219350264832:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 219us (829us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302697.602656:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 786 00000100:00080000:0.0:1789302697.607282:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302697.618693:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302698.640397:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302699.022134:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371095552:0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.022236:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302699.022307:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371096064:0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.022347:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302699.022378:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302699.022384:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371095680:0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.022404:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302699.022442:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095552 00000100:00100000:1.0:1789302699.022452:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095552 00000100:00100000:1.0:1789302699.022524:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095680 00000100:00100000:1.0:1789302699.022528:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095680 00000100:00100000:1.0:1789302699.022567:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:1.0:1789302699.022576:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371095552:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.022644:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371095552:12345-0@lo:400:kworker.0 Request processed in 69us (409us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302699.022652:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:1.0:1789302699.022674:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1789302699.022680:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371095680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.022707:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371095680:12345-0@lo:400:kworker.0 Request processed in 28us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302699.022713:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:1.0:1789302699.022731:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371096064 00000100:00100000:1.0:1789302699.022735:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371096064 00000100:00100000:1.0:1789302699.022781:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 315 00000100:00100000:1.0:1789302699.022789:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371096064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.022821:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371096064:12345-0@lo:400:kworker.0 Request processed in 34us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302699.022828:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 315 00000100:00080000:1.0:1789302699.022853:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302699.022859:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302699.023282:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371095680:0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.023297:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371095552:0@lo:400:kworker.0 00000100:00100000:1.0:1789302699.023306:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371096064:0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.025633:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371095424:0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.025722:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302699.025840:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371095936:0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.025876:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302699.025904:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302699.025911:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371095808:0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.025932:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302699.025966:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095424 00000100:00100000:3.0:1789302699.025974:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095424 00000100:00100000:3.0:1789302699.026021:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:3.0:1789302699.026029:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371095424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.026082:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371095424:12345-0@lo:400:kworker.0 Request processed in 54us (360us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302699.026089:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00100000:3.0:1789302699.026319:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095936 00000100:00100000:3.0:1789302699.026323:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095936 00000100:00100000:3.0:1789302699.026360:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 339 00000100:00100000:3.0:1789302699.026366:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371095936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.026397:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371095936:12345-0@lo:400:kworker.0 Request processed in 32us (521us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302699.026402:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 339 00000100:00100000:3.0:1789302699.026419:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371095808 00000100:00100000:3.0:1789302699.026422:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371095808 00000100:00100000:3.0:1789302699.026448:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:3.0:1789302699.026453:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371095808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.026480:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371095808:12345-0@lo:400:kworker.0 Request processed in 27us (546us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302699.026485:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00080000:3.0:1789302699.026531:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302699.026536:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302699.026576:0:3301:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967334 gen 2 00000004:00080000:3.0:1789302699.026582:0:3301:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967334 committed transno 4294967334 00000100:00100000:3.0:1789302699.026607:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371095808:0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.026616:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371095424:0@lo:400:kworker.0 00000100:00100000:3.0:1789302699.026624:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371095936:0@lo:400:kworker.0 00000100:00100000:2.0:1789302699.284749:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.284869:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265088 00000100:00100000:0.0:1789302699.284883:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265088 00000100:00100000:0.0:1789302699.284980:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:0.0:1789302699.284995:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:2348:x1876219350265088:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302699.285102:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:2348:x1876219350265088:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 111us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302699.285111:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00080000:2.0:1789302699.285384:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302699.297087:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.297130:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.297169:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.297201:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.297248:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265344 00000100:00100000:0.0:1789302699.297269:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265344 00000100:00100000:0.0:1789302699.297347:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265600 00000100:00100000:0.0:1789302699.297350:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265600 00000100:00100000:0.0:1789302699.297370:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265216 00000100:00100000:0.0:1789302699.297372:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265216 00000100:00100000:0.0:1789302699.297391:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 317 00000100:00100000:0.0:1789302699.297405:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350265344:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302699.297529:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350265344:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 127us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302699.297539:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 317 00000100:00100000:0.0:1789302699.297561:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 318 00000100:00100000:0.0:1789302699.297568:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350265600:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302699.297594:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350265600:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302699.297601:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 318 00000100:00100000:0.0:1789302699.297612:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 319 00000100:00100000:0.0:1789302699.297618:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350265216:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302699.297638:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350265216:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302699.297644:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 319 00000100:00100000:0.0:1789302699.297665:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350264960 00000100:00100000:0.0:1789302699.297670:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350264960 00000100:00100000:0.0:1789302699.297696:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 535 00000100:00100000:0.0:1789302699.297703:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350264960:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302699.297729:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350264960:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (560us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302699.297742:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 535 00000100:00080000:0.0:1789302699.297792:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302699.297797:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302699.300767:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.304068:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.304301:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265472 00000100:00100000:0.0:1789302699.304314:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265472 00000100:00100000:0.0:1789302699.304416:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:0.0:1789302699.304431:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2348:x1876219350265472:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302699.304592:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2348:x1876219350265472:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 164us (3850us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302699.304601:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00080000:0.0:1789302699.304637:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302699.304680:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265728 00000100:00100000:0.0:1789302699.304685:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265728 00000100:00100000:0.0:1789302699.304721:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:0.0:1789302699.304728:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350265728:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302699.304778:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350265728:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 50us (732us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302699.304785:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00080000:0.0:1789302699.304803:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:02000400:0.0:1789302699.412974:0:18162:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12b: Restore an imported file implicitly ========================================================== 08:31:37 (1789302697) 00000040:00080000:1.0:1789302699.482092:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302699.482107:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302699.482156:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 19 count 2 00000040:00080000:1.0:1789302699.482165:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302699.533953:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371096192:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302699.534048:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302699.534234:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371096192 00000100:00100000:3.0:1789302699.534246:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371096192 00000100:00100000:3.0:1789302699.534327:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:3.0:1789302699.534339:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371096192:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302699.534390:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 59572224 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302699.534457:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371096192:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (408us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302699.534465:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:1.0:1789302699.535293:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371096192:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302699.671973:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302699.728576:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302699.728821:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265856 00000100:00100000:3.0:1789302699.728843:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265856 00000100:00100000:3.0:1789302699.728925:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 787 00000100:00100000:3.0:1789302699.728937:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23903:x1876219350265856:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1789302699.729073:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23903:x1876219350265856:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 138us (512us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302699.729081:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 787 00000100:00100000:0.0:1789302699.774128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302699.774209:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350265984 00000100:00100000:3.0:1789302699.774223:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350265984 00000100:00100000:3.0:1789302699.774306:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 788 00000100:00100000:3.0:1789302699.774318:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23904:x1876219350265984:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302699.774346:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302699.774396:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cf06 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a00fd expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774484:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774532:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774548:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774564:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774572:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774578:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774865:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774873:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774881:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774889:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302699.774894:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774917:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774933:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:3.0:1789302699.774936:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302699.774943:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774950:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf0d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774964:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774969:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774975:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774980:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302699.774983:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774988:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.774996:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302699.774998:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302699.775001:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.775017:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf14 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.775058:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cf06 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00fd expref: 78 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.775070:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:3.0:1789302699.775072:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302699.775075:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cf06 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a00fd expref: 78 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.775084:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302699.775161:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23904:x1876219350265984:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 846us (1050us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302699.775170:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 788 00000100:00100000:3.0:1789302699.785450:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.787308:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350266240 00000100:00100000:0.0:1789302699.787341:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350266240 00000100:00100000:0.0:1789302699.787548:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 789 00000100:00100000:0.0:1789302699.787560:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23904:x1876219350266240:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:0.0:1789302699.787631:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302699.787661:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184ccf9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a006a expref: 77 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.787685:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302699.787690:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302699.787937:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.787952:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 33 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.787961:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.787999:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 33 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.788014:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 33 pid: 5843 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302699.788027:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371096320:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302699.788131:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599840 going to sleep for 11 seconds 00000100:00100000:1.0:1789302699.794160:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302699.794382:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7b800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371096320:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302699.794803:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302699.795045:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350266368 00000100:00100000:3.0:1789302699.795054:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350266368 00010000:00010000:3.0:1789302699.795088:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 34 pid: 5843 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302699.795229:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 251 00000100:00100000:3.0:1789302699.795241:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2348:x1876219350266368:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302699.795259:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302699.795266:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 34 pid: 5843 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.795294:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 34 pid: 5843 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.795317:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cd23 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0078 expref: 34 pid: 5843 timeout: 636 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.795333:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:3.0:1789302699.795339:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.795348:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.795355:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302699.795419:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302699.795480:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:2348:x1876219350266368:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 241us (1339us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302699.795488:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 251 00010000:00010000:1.0:1789302699.798897:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.798914:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.798947:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.798961:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.798971:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302699.800351:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00010000:00010000:1.0:1789302699.800372:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.800381:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.800388:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.800393:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302699.800398:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.800769:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.800787:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501000) 00010000:00010000:1.0:1789302699.800790:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302699.800795:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302699.800802:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf1b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302699.800881:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000873633f5 reply state 00000000ceeff9ab transno 4294967815 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302699.800890:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302699.801702:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23904:x1876219350266240:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 14139us (16264us total) trans 4294967815 rc 0/0 00000100:00100000:0.0:1789302699.801713:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 789 00010000:00010000:0.0:1789302699.808020:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.808038:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.808045:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.808052:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302699.808056:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.808064:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.808079:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff960f10d24000) 00010000:00010000:0.0:1789302699.808083:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302699.808088:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302699.808095:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184cf22 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302700.045027:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302700.045075:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371096448:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302700.045201:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302700.045717:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371096448 00000100:00100000:2.0:1789302700.045734:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371096448 00000100:00100000:2.0:1789302700.045827:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 145 00000100:00100000:2.0:1789302700.045839:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371096448:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302700.045879:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302700.045981:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371096448:12345-0@lo:13:osp-pre-0-0.0 Request processed in 144us (780us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.045988:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 145 00000100:00100000:1.0:1789302700.046955:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371096448:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302700.064985:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.070364:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350266496 00000100:00100000:0.0:1789302700.070381:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350266496 00000100:00100000:0.0:1789302700.070463:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 790 00000100:00100000:0.0:1789302700.070477:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23941:x1876219350266496:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:0.0:1789302700.070540:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302700.070568:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184cf29 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0104 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.070639:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf30 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.070654:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf30 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.070743:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf30 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302700.070790:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302700.070793:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302700.070812:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf30 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.070844:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184cf29 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0104 expref: 79 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.070862:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf30 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0104 expref: 78 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.070870:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501900) 00010000:00010000:0.0:1789302700.072030:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302700.072035:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221e800, rc 0) 00000100:00100000:0.0:1789302700.072109:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23941:x1876219350266496:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 1632us (7138us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302700.072118:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 790 00000100:00100000:1.0:1789302700.076287:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.076459:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350266752 00000100:00100000:3.0:1789302700.076472:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350266752 00000100:00100000:3.0:1789302700.076589:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 791 00000100:00100000:3.0:1789302700.076603:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23941:x1876219350266752:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:3.0:1789302700.076679:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302700.076713:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf30 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0104 expref: 78 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.076770:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501900) 00010000:00010000:3.0:1789302700.076784:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302700.076987:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23941:x1876219350266752:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 386us (713us total) trans 0 rc -17/-17 00000100:00100000:3.0:1789302700.076994:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 791 00000100:00100000:1.0:1789302700.080559:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.080713:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350266880 00000100:00100000:3.0:1789302700.080726:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350266880 00000100:00100000:3.0:1789302700.080804:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 792 00000100:00100000:3.0:1789302700.080827:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+77:23941:x1876219350266880:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302700.080856:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.080888:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cf37 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a010b expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.080978:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf3e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.080995:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf3e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.081008:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf3e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302700.081066:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302700.081070:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302700.081095:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf3e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.081125:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cf37 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a010b expref: 79 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.081165:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf3e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a010b expref: 78 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.081176:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302700.081179:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302700.081183:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:3.0:1789302700.081267:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23941:x1876219350266880:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 453us (768us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.081274:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 792 00000100:00100000:2.0:1789302700.086733:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.087907:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350267008 00000100:00100000:3.0:1789302700.087926:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350267008 00000100:00100000:3.0:1789302700.088030:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 793 00000100:00100000:3.0:1789302700.088044:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+78:23941:x1876219350267008:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302700.088352:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.088381:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cf45 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0112 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.088470:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.088488:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089325:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089375:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089420:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089430:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089573:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184cf5a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089588:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184cf5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089596:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184cf5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302700.089662:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000004:00200000:3.0:1789302700.089670:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302700.089714:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184cf5a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089814:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089905:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089916:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089928:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.089937:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089963:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.089986:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302700.089993:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302700.090005:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090085:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cf4c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090102:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090109:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090115:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090120:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.090123:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090130:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090138:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302700.090141:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302700.090144:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090162:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cf53 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.090211:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cf45 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0112 expref: 80 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.091136:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184cf5a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0112 expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.091148:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f0cdf2400) 00010000:00010000:3.0:1789302700.091150:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302700.091154:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0000, rc 0) 00000100:00100000:3.0:1789302700.091466:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23941:x1876219350267008:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 3424us (4746us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.091476:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 793 00000100:00100000:1.0:1789302700.160829:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.161167:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350267136 00000100:00100000:0.0:1789302700.161186:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350267136 00000100:00100000:0.0:1789302700.161274:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 794 00000100:00100000:0.0:1789302700.161284:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+33:23947:x1876219350267136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302700.161310:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302700.161337:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184cf61 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0119 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.161414:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.161431:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.161448:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302700.161491:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302700.161495:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302700.161595:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.161634:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184cf61 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 35 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.161687:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.161699:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:0.0:1789302700.161704:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302700.161711:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221e800, rc 0) 00000100:00100000:0.0:1789302700.161919:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23947:x1876219350267136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 531us (1017us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302700.161934:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 794 00000100:00100000:1.0:1789302700.171232:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.171345:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350267392 00000100:00100000:0.0:1789302700.171359:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350267392 00000100:00100000:0.0:1789302700.171442:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 795 00000100:00100000:0.0:1789302700.171455:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23947:x1876219350267392:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302700.171484:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302700.171487:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:0.0:1789302700.171556:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184cf5a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0112 expref: 79 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.171579:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f0cdf2400) 00010000:00010000:0.0:1789302700.171593:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302700.171616:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184cf6f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0120 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.171790:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.171827:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.171839:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.171867:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.171876:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.171883:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175778:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175816:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175834:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302700.175848:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 0 00010000:00010000:0.0:1789302700.175887:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175895:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175903:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175931:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302700.175936:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175947:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175964:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f03501800) 00010000:00010000:0.0:1789302700.175968:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302700.175975:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.175982:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cf76 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302700.176000:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e4e28991 reply state 000000008a72d058 transno 4294967816 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302700.176011:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789302700.176063:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184cf6f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 81 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.176088:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 80 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.176097:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f03501900) 00010000:00010000:0.0:1789302700.176100:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302700.176102:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221ee00, rc 0) 00000100:00100000:0.0:1789302700.176693:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:23947:x1876219350267392:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 5188us (5425us total) trans 4294967816 rc 0/0 00000100:00100000:0.0:1789302700.176715:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 795 00010000:00010000:0.0:1789302700.181027:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.181052:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.181061:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.181068:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302700.181074:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.181084:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.181108:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4f0c1d45 (ffff960f03501e00) 00010000:00010000:0.0:1789302700.181111:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302700.181117:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.181123:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cf7d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.181966:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.182170:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350267648 00000100:00100000:0.0:1789302700.182185:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350267648 00000100:00100000:0.0:1789302700.182255:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 796 00000100:00100000:0.0:1789302700.182274:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:23947:x1876219350267648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302700.182306:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302700.182354:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184cf92 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0127 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.182638:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.182656:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 81 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.182666:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.182713:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 81 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.182729:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 81 pid: 5844 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302700.182744:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371096576:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302700.182870:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599840 going to sleep for 11 seconds 00000100:00100000:2.0:1789302700.185790:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.186478:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350267776 00000100:00100000:1.0:1789302700.186492:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350267776 00010000:00010000:1.0:1789302700.186576:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 82 pid: 5844 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.186629:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 252 00000100:00100000:1.0:1789302700.186641:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:2348:x1876219350267776:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302700.186661:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302700.186670:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 82 pid: 5844 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.186685:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 82 pid: 5844 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.186714:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f03501900) 00010000:00010000:1.0:1789302700.186720:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.186732:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.186741:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.186775:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302700.186857:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:2348:x1876219350267776:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 217us (1080us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.186889:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 252 00010000:00010000:0.0:1789302700.187686:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184cf84 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0120 expref: 81 pid: 5844 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302700.187710:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371096576:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302700.187739:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.187759:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.187779:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302700.187807:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 0 00010000:00010000:0.0:1789302700.188360:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188369:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188377:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188384:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221f200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302700.188389:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188398:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188413:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f03501900) 00010000:00010000:0.0:1789302700.188416:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302700.188422:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188429:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184cf99 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188547:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184cf92 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0127 expref: 81 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188575:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302700.188577:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302700.188583:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184cf92 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0127 expref: 81 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.188596:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221fe00, rc 0) 00000100:00100000:0.0:1789302700.188660:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:23947:x1876219350267648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 6388us (6703us total) trans 4294967817 rc 301/301 00000100:00100000:0.0:1789302700.188669:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 796 00000100:00100000:2.0:1789302700.192952:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.193131:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350268032 00000100:00100000:1.0:1789302700.193145:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350268032 00000100:00100000:1.0:1789302700.193249:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 161 00000100:00100000:1.0:1789302700.193269:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ef050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:23947:x1876219350268032:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302700.193791:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ef050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:23947:x1876219350268032:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 529us (851us total) trans 4294967818 rc 0/0 00000100:00100000:1.0:1789302700.193805:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 161 00000100:00100000:1.0:1789302700.196154:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.196224:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350268160 00000100:00100000:3.0:1789302700.196238:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350268160 00000100:00100000:3.0:1789302700.196351:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 797 00000100:00100000:3.0:1789302700.196364:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:23947:x1876219350268160:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.196396:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.196435:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cfa7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a012e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.196573:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cfa7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a012e expref: 82 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.196590:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cfa7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a012e expref: 82 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.196631:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302700.196634:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302700.196639:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302700.196718:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:23947:x1876219350268160:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 357us (577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.196725:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 797 00000100:00100000:3.0:1789302700.198361:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.198413:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350268416 00000100:00100000:0.0:1789302700.198425:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350268416 00000100:00100000:0.0:1789302700.198484:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 798 00000100:00100000:0.0:1789302700.198494:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:23947:x1876219350268416:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1789302700.198619:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 3/0,1 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.198636:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.198648:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.198990:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.198997:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.199002:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.199006:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302700.199010:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.199018:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.199033:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:1.0:1789302700.199036:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1789302700.199042:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 2/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.199102:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184cfae lrc: 0/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.199196:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:23947:x1876219350268416:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 702us (840us total) trans 4294967819 rc 0/0 00000100:00100000:1.0:1789302700.199205:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 798 00000100:00100000:2.0:1789302700.201709:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.202055:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350268672 00000100:00100000:0.0:1789302700.202068:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350268672 00000100:00100000:0.0:1789302700.202142:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 799 00000100:00100000:0.0:1789302700.202155:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:23947:x1876219350268672:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302700.202622:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.202644:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.202661:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302700.202760:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967820 00000004:00010000:0.0:1789302700.202804:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000001726fe4 reply state 00000000b5384bc1 transno 4294967821 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302700.202814:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302700.202949:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:23947:x1876219350268672:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 796us (1250us total) trans 4294967821 rc 0/0 00000100:00100000:0.0:1789302700.202958:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 799 00000020:00080000:0.0:1789302700.202963:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1789302700.204999:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.205019:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.205026:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.205032:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.205038:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.205046:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.205066:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302700.205069:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789302700.205075:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.205081:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184cfb5 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302700.207033:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.207455:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350268928 00000100:00100000:3.0:1789302700.207469:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350268928 00000100:00100000:3.0:1789302700.207595:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 162 00000100:00100000:3.0:1789302700.207606:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:23947:x1876219350268928:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302700.207990:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:23947:x1876219350268928:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 384us (967us total) trans 4294967822 rc 0/0 00000100:00100000:3.0:1789302700.207999:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 162 00000100:00100000:0.0:1789302700.227167:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.227272:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350269056 00000100:00100000:3.0:1789302700.227286:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350269056 00000100:00100000:3.0:1789302700.227354:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 800 00000100:00100000:3.0:1789302700.227366:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:23950:x1876219350269056:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302700.227411:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.227441:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cfbc lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0135 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227558:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227576:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227590:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227608:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227616:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227646:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227789:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0e00/0x64fe43095184cfd1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227798:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0e00/0x64fe43095184cfd1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227806:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0e00/0x64fe43095184cfd1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302700.227855:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000004:00200000:3.0:1789302700.227861:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302700.227883:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0e00/0x64fe43095184cfd1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227898:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227905:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227913:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227919:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.227924:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227932:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227954:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302700.227957:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302700.227965:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227971:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184cfc3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227986:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227992:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.227998:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.228002:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.228004:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.228009:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.228015:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff960f0cdf2500) 00010000:00010000:3.0:1789302700.228017:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302700.228019:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.228024:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184cfca lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.228073:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cfbc lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0135 expref: 88 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.228088:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0e00/0x64fe43095184cfd1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0135 expref: 87 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.228095:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f0cdf2300) 00010000:00010000:3.0:1789302700.228097:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789302700.228100:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0e00, rc 0) 00000100:00100000:3.0:1789302700.228180:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:23950:x1876219350269056:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 816us (1027us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.228189:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 800 00000100:00100000:0.0:1789302700.232100:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.232228:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350269312 00000100:00100000:3.0:1789302700.232242:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350269312 00000100:00100000:3.0:1789302700.232414:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 801 00000100:00100000:3.0:1789302700.232429:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:23950:x1876219350269312:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302700.232457:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.232486:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184cfd8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a013c expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.232884:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.232902:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.232917:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.232934:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.232939:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.232943:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233090:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cfed lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233099:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cfed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233115:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cfed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302700.233128:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:2.0:1789302700.233256:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233470:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233478:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233484:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.233489:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233499:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233587:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f0cdf2300) 00010000:00010000:2.0:1789302700.233590:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302700.233598:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233605:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184cfdf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233622:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233639:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233645:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233650:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.233653:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233659:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233667:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4f0c1d45 (ffff960f03927f00) 00010000:00010000:2.0:1789302700.233669:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302700.233672:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233677:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184cfe6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233733:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184cfd8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a013c expref: 90 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233760:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cfed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a013c expref: 89 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.233769:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:2.0:1789302700.233771:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302700.233780:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe400, rc 0) 00000100:00100000:2.0:1789302700.233871:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23950:x1876219350269312:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1445us (1790us total) trans 4294967823 rc 0/0 00000100:00100000:2.0:1789302700.233880:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 801 00000100:00100000:2.0:1789302700.238714:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.238805:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350269440 00000100:00100000:3.0:1789302700.238819:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350269440 00000100:00100000:3.0:1789302700.238904:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 802 00000100:00100000:3.0:1789302700.238916:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23950:x1876219350269440:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:3.0:1789302700.239008:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239058:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239073:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239109:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239117:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239126:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239133:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.239138:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239147:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239161:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302700.239165:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302700.239171:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.239177:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184cffb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.239265:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23950:x1876219350269440:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 351us (565us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.239280:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 802 00000100:00100000:0.0:1789302700.241835:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.241950:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350269696 00000100:00100000:1.0:1789302700.241963:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350269696 00000100:00100000:1.0:1789302700.242341:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 163 00000100:00100000:1.0:1789302700.242356:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23950:x1876219350269696:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:1.0:1789302700.243240:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:23950:x1876219350269696:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 882us (1411us total) trans 4294967824 rc 0/0 00000100:00100000:1.0:1789302700.243259:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 163 00000100:00100000:0.0:1789302700.295063:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.298695:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350269952 00000100:00100000:1.0:1789302700.298718:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350269952 00000100:00100000:1.0:1789302700.298833:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 803 00000100:00100000:1.0:1789302700.298846:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:23955:x1876219350269952:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302700.298884:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302700.298917:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d002 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0143 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.299060:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d009 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.299079:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d009 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.299118:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d009 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302700.299132:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:1.0:1789302700.299609:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d002 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0143 expref: 93 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.299639:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d009 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0143 expref: 92 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.299649:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:1.0:1789302700.299653:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:1.0:1789302700.299657:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:1.0:1789302700.299739:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:23955:x1876219350269952:12345-192.168.203.11@tcp:101:cat.0 Request processed in 896us (4690us total) trans 4294967825 rc 0/0 00000100:00100000:1.0:1789302700.299758:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 803 00000100:00100000:3.0:1789302700.305637:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.305741:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350270208 00000100:00100000:0.0:1789302700.305765:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350270208 00000100:00100000:0.0:1789302700.305867:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 33 00000100:00100000:0.0:1789302700.305880:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:23955:x1876219350270208:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:0.0:1789302700.305903:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789302700.305938:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184cfed lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a013c expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.305995:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d009 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0143 expref: 91 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.306009:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184cfa7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a012e expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.306020:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:0.0:1789302700.306034:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302700.306110:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b3c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23955:x1876219350270208:12345-192.168.203.11@tcp:103:cat.0 Request processed in 233us (488us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302700.306118:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 33 00000100:00100000:0.0:1789302700.308856:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.308990:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350270336 00000100:00100000:3.0:1789302700.309003:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350270336 00000100:00100000:3.0:1789302700.309066:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 804 00000100:00100000:3.0:1789302700.309079:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23955:x1876219350270336:12345-192.168.203.11@tcp:58:cat.0 00010000:00010000:3.0:1789302700.309248:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.309268:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.309284:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302700.309590:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].19, 136 off10776 00000100:00100000:3.0:1789302700.309706:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23955:x1876219350270336:12345-192.168.203.11@tcp:58:cat.0 Request processed in 627us (861us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.309715:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 804 00000100:00100000:2.0:1789302700.313369:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.313544:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350270464 00000100:00100000:1.0:1789302700.313560:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350270464 00000100:00100000:1.0:1789302700.313639:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 805 00000100:00100000:1.0:1789302700.313652:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:23955:x1876219350270464:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302700.313680:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302700.313710:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a014a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.313798:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.313844:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 4/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a014a expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.313865:0:5845:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.313880:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 3/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a014a expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.313897:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:1.0:1789302700.313900:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302700.313904:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 3/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a014a expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.313911:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea200, rc 0) 00000100:00100000:1.0:1789302700.313990:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:23955:x1876219350270464:12345-192.168.203.11@tcp:101:cat.0 Request processed in 341us (638us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.313998:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 805 00000040:00080000:1.0:1789302700.493166:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302700.493180:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302700.493228:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 20 count 3 00000040:00080000:1.0:1789302700.493237:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302700.493407:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63ed80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371096704:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302700.493624:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03f94840 going to sleep for 11 seconds 00000100:00100000:2.0:1789302700.502740:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.503554:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350270592 00000100:00100000:3.0:1789302700.503568:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350270592 00000100:00100000:3.0:1789302700.503657:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 806 00000100:00100000:3.0:1789302700.503669:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23956:x1876219350270592:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302700.503739:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302700.504145:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23956:x1876219350270592:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 477us (1419us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.504154:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 806 00000100:00100000:1.0:1789302700.507928:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63ed80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371096704:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302700.512206:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.512839:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350270720 00000100:00100000:1.0:1789302700.512872:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350270720 00000100:00100000:1.0:1789302700.512961:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 807 00000100:00100000:1.0:1789302700.512974:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23956:x1876219350270720:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1789302700.513201:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513226:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513243:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513270:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513278:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513285:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513315:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513324:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513331:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302700.513380:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00000004:00200000:1.0:1789302700.513385:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302700.513583:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513598:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513605:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513613:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513619:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302700.513625:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513633:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513648:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:1.0:1789302700.513651:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302700.513656:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513663:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d025 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513679:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513685:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513691:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513703:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302700.513706:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513713:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513721:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3d0032c8 (ffff960f10d24d00) 00010000:00010000:1.0:1789302700.513724:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302700.513727:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513732:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d02c lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3d0032c8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513764:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513772:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513778:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513783:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302700.513786:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513793:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513811:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:1.0:1789302700.513813:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302700.513817:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.513823:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d033 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.513926:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23956:x1876219350270720:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 953us (1733us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.513936:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 807 00000100:00100000:0.0:1789302700.519918:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.520654:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350270976 00000100:00100000:2.0:1789302700.520671:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350270976 00000100:00100000:2.0:1789302700.520802:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 808 00000100:00100000:2.0:1789302700.520814:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:23956:x1876219350270976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302700.520938:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302700.521089:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d03a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0151 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.521222:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.521242:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.521334:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302700.521350:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:2.0:1789302700.523137:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523155:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523183:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523190:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.523195:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523204:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523225:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:2.0:1789302700.523229:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302700.523235:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523242:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d041 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523311:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d03a lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0151 expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523323:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302700.523325:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302700.523329:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d03a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0151 expref: 36 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.523340:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe800, rc 0) 00000100:00100000:2.0:1789302700.523678:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23956:x1876219350270976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2864us (3773us total) trans 4294967826 rc 301/301 00000100:00100000:2.0:1789302700.523698:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 808 00000100:00100000:0.0:1789302700.527673:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.527790:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350271104 00000100:00100000:2.0:1789302700.527803:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350271104 00000100:00100000:2.0:1789302700.528042:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 809 00000100:00100000:2.0:1789302700.528056:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23956:x1876219350271104:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302700.528106:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302700.528287:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23956:x1876219350271104:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 234us (628us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.528295:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 809 00000100:00100000:0.0:1789302700.531190:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.531363:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350271232 00000100:00100000:3.0:1789302700.531377:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350271232 00000100:00100000:3.0:1789302700.531611:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 810 00000100:00100000:3.0:1789302700.531624:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23956:x1876219350271232:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.531650:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.531675:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d04f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0158 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531764:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531782:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531795:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531810:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531817:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531829:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531966:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0800/0x64fe43095184d064 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.531974:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0800/0x64fe43095184d064 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532004:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0800/0x64fe43095184d064 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302700.532041:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x33:0x0]: returning size 16384 00000004:00200000:3.0:1789302700.532045:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302700.532067:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0800/0x64fe43095184d064 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532081:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532088:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532096:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532102:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.532107:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532116:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532130:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302700.532133:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302700.532139:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532146:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d056 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532175:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532182:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532188:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532194:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.532197:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532203:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532211:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa5954980 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302700.532214:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302700.532217:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532223:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d05d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa5954980 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532260:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d04f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0158 expref: 37 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532276:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0800/0x64fe43095184d064 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0158 expref: 36 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.532296:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f0cdf2300) 00010000:00010000:3.0:1789302700.532299:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789302700.532302:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0800, rc 0) 00000100:00100000:3.0:1789302700.533769:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23956:x1876219350271232:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2137us (2584us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.533781:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 810 00000100:00100000:0.0:1789302700.538585:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.538844:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350271488 00000100:00100000:3.0:1789302700.538859:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350271488 00000100:00100000:3.0:1789302700.538923:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 811 00000100:00100000:3.0:1789302700.538934:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:23956:x1876219350271488:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.538972:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.539002:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d06b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a015f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302700.539963:0:5845:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x10:0x0], index 1: 0 00010000:00010000:3.0:1789302700.540282:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d072 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.540302:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d072 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.540317:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d072 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.540395:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d06b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a015f expref: 39 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.540432:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d072 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a015f expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.540443:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302700.540447:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302700.540451:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302700.540568:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23956:x1876219350271488:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1636us (1996us total) trans 4294967827 rc 0/0 00000100:00100000:3.0:1789302700.540578:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 811 00000100:00100000:1.0:1789302700.544615:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.544736:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350271744 00000100:00100000:2.0:1789302700.544767:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350271744 00000100:00100000:2.0:1789302700.544844:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 812 00000100:00100000:2.0:1789302700.544856:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23956:x1876219350271744:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302700.544933:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302700.544968:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d072 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a015f expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.544991:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:2.0:1789302700.545015:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302700.545045:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.545062:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.545074:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546460:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546484:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546496:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546587:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546606:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546613:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546620:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.546624:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.546632:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548840:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f0cdf2300) 00010000:00010000:2.0:1789302700.548844:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302700.548852:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548860:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d080 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548897:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548904:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548911:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548916:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.548920:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548929:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548937:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x7349e6ea (ffff960f03927f00) 00010000:00010000:2.0:1789302700.548939:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302700.548942:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.548947:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d087 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x33:0x0].0x7349e6ea bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302700.549071:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23956:x1876219350271744:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 4213us (4465us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302700.549082:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 812 00000100:00100000:2.0:1789302700.551764:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.551860:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350271872 00000100:00100000:1.0:1789302700.551877:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350271872 00000100:00100000:1.0:1789302700.551955:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 813 00000100:00100000:1.0:1789302700.551967:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23956:x1876219350271872:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302700.551995:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302700.552025:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d08e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0166 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.552079:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.552096:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.552110:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.552247:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d08e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.552278:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 38 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.552288:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302700.552291:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302700.552295:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb200, rc 0) 00000100:00100000:1.0:1789302700.552381:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23956:x1876219350271872:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 415us (635us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.552389:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 813 00000100:00100000:2.0:1789302700.554545:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.554703:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350272128 00000100:00100000:3.0:1789302700.554720:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350272128 00000100:00100000:3.0:1789302700.554806:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 814 00000100:00100000:3.0:1789302700.554827:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23956:x1876219350272128:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.554935:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.554956:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.554969:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302700.555353:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].7, 88 off8712 00000004:00080000:3.0:1789302700.555367:0:5845:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 537565863518ns, next at 538565859606ns, rc = 0 00000004:00010000:3.0:1789302700.555471:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000023a98580 reply state 00000000232b04a5 transno 4294967828 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.555543:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302700.555669:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23956:x1876219350272128:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 852us (1164us total) trans 4294967828 rc 0/0 00000100:00100000:3.0:1789302700.555678:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 814 00000100:00100000:2.0:1789302700.558893:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:0.0:1789302700.559108:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.559123:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.559132:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.559139:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302700.559145:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.559162:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.559183:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:0.0:1789302700.559186:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789302700.559192:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.559199:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184d09c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.559360:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350272256 00000100:00100000:3.0:1789302700.559372:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350272256 00000100:00100000:3.0:1789302700.559465:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 815 00000100:00100000:3.0:1789302700.559478:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23956:x1876219350272256:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.559535:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302700.559557:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a016d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.559635:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.559651:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.559671:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:3.0:1789302700.559674:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302700.559677:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302700.559761:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:23956:x1876219350272256:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 286us (882us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.559767:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 815 00000100:00100000:2.0:1789302700.564338:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.564705:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350272384 00000100:00100000:0.0:1789302700.564717:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350272384 00000100:00100000:0.0:1789302700.564848:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 197 00000100:00100000:0.0:1789302700.564860:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350272384:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1789302700.565346:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350272384:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 486us (1019us total) trans 4294967334 rc 0/0 00000100:00100000:0.0:1789302700.565355:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 197 00000100:00100000:0.0:1789302700.569684:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.569774:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350272640 00000100:00100000:3.0:1789302700.569789:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350272640 00000100:00100000:3.0:1789302700.569893:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 164 00000100:00100000:3.0:1789302700.569905:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:23956:x1876219350272640:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302700.570235:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23956:x1876219350272640:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 331us (565us total) trans 4294967829 rc 0/0 00000100:00100000:3.0:1789302700.570244:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 164 00000100:00100000:2.0:1789302700.574154:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.574264:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350272896 00000100:00100000:0.0:1789302700.574278:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350272896 00000100:00100000:0.0:1789302700.574354:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 816 00000100:00100000:0.0:1789302700.574366:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23956:x1876219350272896:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302700.574551:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23956:x1876219350272896:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 187us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302700.574559:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 816 00000100:00100000:0.0:1789302700.577645:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.577772:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350273024 00000100:00100000:2.0:1789302700.577786:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350273024 00000100:00100000:2.0:1789302700.577871:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 817 00000100:00100000:2.0:1789302700.577882:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23956:x1876219350273024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302700.577912:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302700.577939:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d0aa lrc: 2/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0174 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.578035:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d0b1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.578053:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d0b1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.578068:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d0b1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302700.578115:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302700.578138:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d0b1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.578169:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d0aa lrc: 0/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0174 expref: 43 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.578190:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d0b1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0174 expref: 42 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.578200:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:2.0:1789302700.578211:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1789302700.578216:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbee00, rc 0) 00000100:00100000:2.0:1789302700.578309:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23956:x1876219350273024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 428us (678us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.578317:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 817 00000100:00100000:3.0:1789302700.585833:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.585972:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350273152 00000100:00100000:2.0:1789302700.585986:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350273152 00000100:00100000:2.0:1789302700.586052:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 198 00000100:00100000:2.0:1789302700.586064:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350273152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302700.586088:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302700.586114:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a017b expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302700.586150:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x10:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6aa697ac, blocks: 0x1 00010000:00010000:2.0:1789302700.586179:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=537s v=(0 0) 00010000:00010000:2.0:1789302700.586184:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302700.586196:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.586218:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.586233:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe800, rc 0) 00000100:00100000:2.0:1789302700.586319:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350273152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 257us (501us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.586326:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 198 00000100:00100000:0.0:1789302700.591280:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.591372:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350273408 00000100:00100000:3.0:1789302700.591386:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350273408 00000100:00100000:3.0:1789302700.591462:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 818 00000100:00100000:3.0:1789302700.591474:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23956:x1876219350273408:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302700.591649:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23956:x1876219350273408:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 177us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.591657:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 818 00000100:00100000:1.0:1789302700.594965:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.598304:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350273536 00000100:00100000:0.0:1789302700.598324:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350273536 00000100:00100000:0.0:1789302700.598548:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 199 00000100:00100000:0.0:1789302700.598563:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:23956:x1876219350273536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302700.598589:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302700.598615:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 2/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0182 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.598660:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.598669:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302700.598674:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302700.598681:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 4/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.601148:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.601182:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 12 pid: 6697 timeout: 637 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302700.601204:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371096832:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302700.601587:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599600 going to sleep for 11 seconds 00000100:00100000:0.0:1789302700.606822:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.606983:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7b100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371096832:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302700.607030:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 3/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.607057:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221fc00, rc 0) 00000100:00100000:2.0:1789302700.607150:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:23956:x1876219350273536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 8590us (12199us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.607161:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 199 00000100:00100000:3.0:1789302700.612926:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350273664 00000100:00100000:3.0:1789302700.612957:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350273664 00010000:00010000:3.0:1789302700.613055:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 12 pid: 6697 timeout: 637 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.613294:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:3.0:1789302700.613321:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350273664:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302700.613350:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302700.613700:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 12 pid: 6697 timeout: 637 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.613730:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 12 pid: 6697 timeout: 637 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.613791:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000014cf43c3/0x64fe43095184d0b8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a017b expref: 12 pid: 6697 timeout: 637 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.613825:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302700.613830:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302700.613840:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.613852:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.613908:0:8463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.613925:0:8463:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8463:1876219371096960:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302700.614273:0:8463:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302700.618617:0:8463:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8463:1876219371096960:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302700.618660:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302700.619993:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.622869:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350273792 00000100:00100000:3.0:1789302700.622906:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350273792 00002000:00100000:3.0:1789302700.623041:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x10:0x0] (0->157) req@ffff960f25c45500 x1876219350273792/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:276/0 lens 648/0 e 0 to 0 dl 1789302711 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302700.623064:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25c45500 with x1876219350273792 ext(0->157) 00002000:00010000:3.0:1789302700.623072:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25c45500 00000100:00100000:3.0:1789302700.624659:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 65 00000100:00100000:3.0:1789302700.624694:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350273792:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302700.625296:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350273664:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 11989us (18490us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.625308:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:0.0:1789302700.628732:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350273792:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 4040us (8724us total) trans 4294967335 rc 0/0 00000100:00100000:0.0:1789302700.628760:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 65 00002000:00100000:0.0:1789302700.628777:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x10:0x0] (0->157) req@ffff960f25c45500 x1876219350273792/t4294967335(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:276/0 lens 648/448 e 0 to 0 dl 1789302711 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302700.628809:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25c45500 with x1876219350273792 ext(0->157) 00002000:00010000:0.0:1789302700.628827:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25c45500 00000100:00100000:0.0:1789302700.634705:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.634897:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350273920 00000100:00100000:2.0:1789302700.634912:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350273920 00000100:00100000:2.0:1789302700.635346:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 819 00000100:00100000:2.0:1789302700.635360:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:23956:x1876219350273920:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.685666:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302700.703530:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967821 is committed 00000001:00080000:1.0:1789302700.703573:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967825 is committed 00000001:00080000:1.0:1789302700.703589:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967826 is committed 00000020:00080000:1.0:1789302700.703603:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 11 batchid updated 00000001:00080000:1.0:1789302700.703627:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967829 is committed 00000004:00080000:1.0:1789302700.703635:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 8 committed 00000100:00100000:0.0:1789302700.703785:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23956:x1876219350273920:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 68420us (69086us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302700.703804:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 819 00000100:00100000:1.0:1789302700.703845:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371097088:0@lo:2:osp-syn-1-0.0 00000100:00100000:1.0:1789302700.703926:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302700.704000:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371097088 00000100:00100000:0.0:1789302700.704022:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371097088 00000100:00100000:0.0:1789302700.704083:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:0.0:1789302700.704093:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371097088:12345-0@lo:2:osp-syn-1-0.0 00002000:00010000:0.0:1789302700.704285:0:6695:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:0.0:1789302700.704289:0:6695:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb mtime from disk: 0 -> 1789302700 00002000:00010000:0.0:1789302700.704292:0:6695:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:0.0:1789302700.704294:0:6695:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb ctime from disk: 1789302700 -> 1789302700 00002000:00010000:0.0:1789302700.704296:0:6695:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:0.0:1789302700.704356:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371097088:12345-0@lo:2:osp-syn-1-0.0 Request processed in 264us (430us total) trans 4294967336 rc 0/0 00000100:00100000:0.0:1789302700.704363:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000004:00080000:0.0:1789302700.704411:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f63dc00/2, rc 0, transno 4294967336 00000100:00100000:0.0:1789302700.704417:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371097088:0@lo:2:osp-syn-1-0.0 00000020:00080000:3.0:1789302700.705650:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 11 00000020:00080000:3.0:1789302700.705674:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 11 00000100:00100000:2.0:1789302700.705844:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.705928:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350274048 00000100:00100000:0.0:1789302700.705939:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350274048 00000100:00100000:0.0:1789302700.705995:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 201 00000100:00100000:0.0:1789302700.706005:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2345:x1876219350274048:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000001:00080000:1.0:1789302700.762899:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967336 is committed 00000001:00080000:1.0:1789302700.762920:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967334 is committed 00000001:00080000:1.0:1789302700.762929:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967335 is committed 00000100:00100000:2.0:1789302700.764829:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350274048:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 58800us (58964us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.764867:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 201 00000100:00100000:0.0:1789302700.767318:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.767574:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350274304 00000100:00100000:3.0:1789302700.767587:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350274304 00000100:00100000:3.0:1789302700.767664:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 820 00000100:00100000:3.0:1789302700.767676:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23956:x1876219350274304:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.767756:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302700.767791:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d0b1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0174 expref: 38 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.767826:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:3.0:1789302700.767833:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302700.767874:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.767891:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.767903:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302700.768166:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ae4cb7ae reply state 000000005450637c transno 4294967830 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.768177:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302700.768310:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:23956:x1876219350274304:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 635us (1007us total) trans 4294967830 rc 0/0 00000100:00100000:3.0:1789302700.768318:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 820 00010000:00010000:0.0:1789302700.772898:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.772923:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.772933:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.772940:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302700.772946:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.772956:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.772977:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:0.0:1789302700.772981:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302700.772987:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.772995:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d0c6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302700.773989:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.774127:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350274432 00000100:00100000:2.0:1789302700.774141:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350274432 00000100:00100000:2.0:1789302700.774218:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 202 00000100:00100000:2.0:1789302700.774231:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350274432:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1789302700.775061:0:6696:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:2.0:1789302700.775067:0:6696:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb mtime from disk: 1789302700 -> 1536580263 00002000:00010000:2.0:1789302700.775071:0:6696:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb atime from disk: 0 -> 1789302187 00002000:00010000:2.0:1789302700.775073:0:6696:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb ctime from disk: 1789302700 -> 1789302699 00002000:00010000:2.0:1789302700.775076:0:6696:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x10:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1789302700.775206:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350274432:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 975us (1223us total) trans 4294967337 rc 0/0 00000100:00100000:2.0:1789302700.775218:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 202 00000100:00100000:3.0:1789302700.780976:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302700.781056:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350274688 00000100:00100000:1.0:1789302700.781070:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350274688 00000100:00100000:1.0:1789302700.781145:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 821 00000100:00100000:1.0:1789302700.781172:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:23956:x1876219350274688:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1789302700.781264:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.781284:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 38 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.781295:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 4/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.781341:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 38 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.781357:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 38 pid: 6919 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.781374:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371097216:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302700.781773:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03f94480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302700.787080:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.787227:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371097216:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302700.787266:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.787361:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350274816 00000100:00100000:1.0:1789302700.787370:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350274816 00010000:00010000:1.0:1789302700.787418:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 39 pid: 6919 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.787466:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:1.0:1789302700.787477:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2348:x1876219350274816:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302700.787495:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302700.787640:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 39 pid: 6919 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.787681:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 39 pid: 6919 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.787709:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d095 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x15:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0166 expref: 39 pid: 6919 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.787728:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302700.787734:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.787755:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.787764:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.787793:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302700.787922:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.787931:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.787971:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.787978:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.787985:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.787992:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302700.787997:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.788006:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.788015:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:3.0:1789302700.788018:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1789302700.788022:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 2/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.788041:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d0cd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.788107:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:2348:x1876219350274816:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 631us (1041us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.788117:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:3.0:1789302700.793881:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23956:x1876219350274688:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 12690us (12899us total) trans 0 rc -22/-22 00000100:00100000:3.0:1789302700.793905:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 821 00000100:00100000:0.0:1789302700.805657:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.806325:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350274944 00000100:00100000:2.0:1789302700.806341:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350274944 00000100:00100000:2.0:1789302700.806472:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 203 00000100:00100000:2.0:1789302700.806486:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350274944:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302700.806605:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.806634:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.806669:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302700.806674:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302700.806680:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.806728:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.806756:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 11 pid: 6696 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302700.806774:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ac380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371097344:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302700.807020:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0c757cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302700.815986:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.816223:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ac380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371097344:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302700.816261:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.816387:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350275072 00000100:00100000:3.0:1789302700.816397:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350275072 00010000:00010000:3.0:1789302700.816440:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 12 pid: 6696 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.816489:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 255 00000100:00100000:3.0:1789302700.816530:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350275072:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302700.816550:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302700.816581:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 12 pid: 6696 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.816597:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 12 pid: 6696 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.816624:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000abf7a099/0x64fe43095184d0bf lrc: 0/0,0 mode: --/PW res: [0x280000400:0x10:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0182 expref: 12 pid: 6696 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.816648:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302700.816653:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302700.816662:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.816695:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.816727:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302700.816799:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350275072:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 300us (829us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.816807:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 255 00010000:00010000:2.0:1789302700.816910:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.816921:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.816948:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.816963:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.816971:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.816978:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302700.816983:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.816992:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.817006:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.817013:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aea66f0b/0x64fe43095184d0d4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302700.817065:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350274944:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 10583us (11425us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.817073:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 203 00000100:00100000:2.0:1789302700.819197:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.819423:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350275328 00000100:00100000:3.0:1789302700.819436:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350275328 00000100:00100000:3.0:1789302700.819536:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 822 00000100:00100000:3.0:1789302700.819549:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:23956:x1876219350275328:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302700.819657:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 3/0,1 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.819681:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.819695:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.820029:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.820043:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.820053:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.820115:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.820134:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 38 pid: 5845 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.820151:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371097472:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302700.831167:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.831474:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cc54180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302700.831480:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350275456 00000100:00100000:3.0:1789302700.831491:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350275456 00000100:00100000:2.0:1789302700.831540:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371097472:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302700.831564:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 39 pid: 5845 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.831589:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302700.831599:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 256 00000100:00100000:3.0:1789302700.831610:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2347:x1876219350275456:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302700.831629:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302700.831639:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 39 pid: 5845 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.831653:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 39 pid: 5845 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.831683:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d0a3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a016d expref: 39 pid: 5845 timeout: 637 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.831706:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:3.0:1789302700.831711:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.831728:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.831736:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.831778:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302700.831860:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:2347:x1876219350275456:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 251us (707us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302700.831869:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 256 00010000:00010000:2.0:1789302700.832888:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.832902:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302700.833180:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967831 00000001:00080000:2.0:1789302700.833950:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967832 00000001:00080000:2.0:1789302700.834083:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967833 00000100:00100000:2.0:1789302700.834199:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371097600:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1789302700.834639:0:5843:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302700.834688:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58af00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302700.835112:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371097600 00000100:00100000:1.0:1789302700.835123:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371097600 00000100:00100000:1.0:1789302700.835197:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 27 00000100:00100000:1.0:1789302700.835210:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371097600:12345-0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1789302700.835922:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371097600:12345-0@lo:1000:mdt00_000.0 Request processed in 712us (1280us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.835950:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 27 00000100:00100000:2.0:1789302700.836727:0:5843:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967336 gen 2 00000004:00080000:2.0:1789302700.836736:0:5843:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f63dc00, transno 4294967336 00000100:00100000:2.0:1789302700.836768:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371097600:0@lo:1000:mdt00_000.0 00000001:00080000:2.0:1789302700.838113:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967834 00000100:00100000:2.0:1789302700.838223:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371097728:0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1789302700.838320:0:5843:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302700.838377:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0c757cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302700.838564:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371097728 00000100:00100000:0.0:1789302700.838575:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371097728 00000100:00100000:0.0:1789302700.838641:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 28 00000100:00100000:0.0:1789302700.838653:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371097728:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1789302700.838921:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5843:x1876219371097728:12345-0@lo:1000:mdt00_000.0 Request processed in 269us (600us total) trans 4294967338 rc 0/0 00000100:00100000:0.0:1789302700.838939:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 28 00000100:00100000:2.0:1789302700.839048:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095adc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371097728:0@lo:1000:mdt00_000.0 00010000:00010000:2.0:1789302700.839099:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839111:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839132:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839143:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.839150:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839160:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839183:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:2.0:1789302700.839187:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302700.839193:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839200:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184d0e2 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x15:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839240:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839247:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839253:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839271:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.839274:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839291:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839300:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:2.0:1789302700.839302:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302700.839311:0:5843:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a014a expref: 78 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839324:0:5843:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a014a expref: 78 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839589:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a014a expref: 78 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.839630:0:5843:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea200/0x64fe43095184d01e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a014a expref: 78 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302700.839646:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371097856:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302700.839756:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10843780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302700.850571:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371097856:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302700.850593:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302700.850619:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850633:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184d017 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850680:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850722:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850738:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850763:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850770:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850777:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850783:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00000100:00100000:1.0:1789302700.850784:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350275584 00010000:00010000:2.0:1789302700.850788:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.850793:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350275584 00010000:00010000:2.0:1789302700.850796:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850808:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:2.0:1789302700.850812:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302700.850817:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850823:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d0e9 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x35:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850832:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850844:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850850:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850855:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.850858:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.850862:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 165 00010000:00010000:2.0:1789302700.850864:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850872:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00000100:00100000:1.0:1789302700.850873:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23955:x1876219350275584:12345-192.168.203.11@tcp:49:cat.0 00010000:00010000:2.0:1789302700.850874:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302700.850877:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 2/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.850882:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184d0db lrc: 0/0,0 mode: --/PW res: [0x200000401:0x35:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302700.851043:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23955:x1876219350275584:12345-192.168.203.11@tcp:49:cat.0 Request processed in 171us (451us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.851051:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 165 00000100:00100000:2.0:1789302700.851146:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23956:x1876219350275328:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 31600us (31961us total) trans 4294967835 rc 0/0 00000100:00100000:2.0:1789302700.851156:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 822 00000020:00080000:2.0:1789302700.851161:0:5843:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:1.0:1789302700.851327:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302700.851333:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302700.851900:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1789302700.851912:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302700.859376:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.859432:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.859555:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350275840 00000100:00100000:2.0:1789302700.859569:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350275840 00000100:00100000:2.0:1789302700.859642:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 166 00000100:00100000:2.0:1789302700.859654:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:23956:x1876219350275840:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302700.860646:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350275968 00000100:00100000:1.0:1789302700.860655:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350275968 00000100:00100000:1.0:1789302700.860710:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 204 00000100:00100000:1.0:1789302700.860720:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:23955:x1876219350275968:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302700.860733:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302700.860771:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c17c244b/0x64fe43095184d0f0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0189 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302700.860804:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x10:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6aa695ab, ctime: 0x6aa697ab, blocks: 0xb 00010000:00010000:1.0:1789302700.860941:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=537s v=(0 0) 00010000:00010000:1.0:1789302700.860947:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x10:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302700.860960:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c17c244b/0x64fe43095184d0f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0189 expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.860997:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c17c244b/0x64fe43095184d0f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x10:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0189 expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302700.861013:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea000, rc 0) 00000100:00100000:1.0:1789302700.861097:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:23955:x1876219350275968:12345-192.168.203.11@tcp:101:cat.0 Request processed in 377us (1662us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302700.861106:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 204 00000004:00080000:0.0:1789302700.864227:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x15:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1789302700.864389:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967836 00010000:00010000:0.0:1789302700.864493:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.864552:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.864568:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.864579:0:5846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.864586:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.864592:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.864624:0:5846:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302700.864781:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:23956:x1876219350275840:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 5123us (5415us total) trans 4294967837 rc 0/0 00000100:00100000:0.0:1789302700.864793:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 166 00000020:00080000:0.0:1789302700.864801:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1789302700.864984:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302700.864995:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.865003:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221f200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302700.865007:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.865017:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.865033:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x15:0x0].0x0 (ffff960f03501800) 00010000:00010000:3.0:1789302700.865037:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302700.865043:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.865049:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d0f7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x15:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302700.865085:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302700.865087:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302700.865089:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302700.871770:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.874733:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350276096 00000100:00100000:2.0:1789302700.874766:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350276096 00002000:00100000:0.0:1789302700.875002:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x10:0x0] (0->4095) req@ffff960f3f63dc00 x1876219350276096/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:276/0 lens 488/0 e 0 to 0 dl 1789302711 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302700.875041:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63dc00 with x1876219350276096 ext(0->4095) 00002000:00010000:0.0:1789302700.875062:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f63dc00 00000100:00100000:0.0:1789302700.875106:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 66 00000100:00100000:0.0:1789302700.875117:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350276096:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:0.0:1789302700.875686:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350276096:12345-192.168.203.11@tcp:3:cat.0 Request processed in 566us (3945us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302700.875696:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 66 00002000:00100000:0.0:1789302700.875705:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x10:0x0] (0->4095) req@ffff960f3f63dc00 x1876219350276096/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:276/0 lens 488/4536 e 0 to 0 dl 1789302711 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302700.875715:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f63dc00 with x1876219350276096 ext(0->4095) 00002000:00010000:0.0:1789302700.875719:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f63dc00 00000100:00100000:1.0:1789302700.880820:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302700.881070:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350276352 00000100:00100000:3.0:1789302700.881085:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350276352 00000100:00100000:3.0:1789302700.881159:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 167 00000100:00100000:3.0:1789302700.881172:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ef850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+79:23955:x1876219350276352:12345-192.168.203.11@tcp:35:cat.0 00000001:00080000:3.0:1789302700.881603:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967838 00000100:00100000:3.0:1789302700.881756:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ef850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:23955:x1876219350276352:12345-192.168.203.11@tcp:35:cat.0 Request processed in 574us (939us total) trans 4294967839 rc 0/0 00000100:00100000:3.0:1789302700.881765:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 167 00000020:00080000:3.0:1789302700.881771:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789302700.925837:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.927149:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350276608 00000100:00100000:0.0:1789302700.927167:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350276608 00000100:00100000:0.0:1789302700.927238:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 823 00000100:00100000:0.0:1789302700.927252:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:23960:x1876219350276608:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302700.927290:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302700.927321:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d0fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0190 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927441:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927470:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927542:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927561:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927569:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927575:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927981:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d113 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.927992:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d113 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928000:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d113 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302700.928028:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:0.0:1789302700.928171:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928180:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928188:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928195:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302700.928209:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928219:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928234:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x0 (ffff960f0cdf2300) 00010000:00010000:0.0:1789302700.928238:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302700.928244:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928252:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d105 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928268:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928274:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928281:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928286:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221f400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302700.928289:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928295:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928303:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x33:0x0].0x4f0c1d45 (ffff960f03501000) 00010000:00010000:0.0:1789302700.928305:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302700.928317:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928323:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d10c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x33:0x0].0x4f0c1d45 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928378:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d0fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0190 expref: 84 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928394:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f200/0x64fe43095184d113 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0190 expref: 83 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302700.928402:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:0.0:1789302700.928404:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302700.928407:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f200, rc 0) 00000100:00100000:0.0:1789302700.928499:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:23960:x1876219350276608:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1251us (2678us total) trans 4294967840 rc 0/0 00000100:00100000:0.0:1789302700.928567:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 823 00000100:00100000:1.0:1789302700.931960:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302700.932168:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350276736 00000100:00100000:2.0:1789302700.932183:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350276736 00000100:00100000:2.0:1789302700.932257:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 824 00000100:00100000:2.0:1789302700.932270:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:23960:x1876219350276736:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:2.0:1789302700.932379:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932402:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932418:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932454:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932462:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932469:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932475:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302700.932481:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932490:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932578:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:2.0:1789302700.932581:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302700.932587:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302700.932594:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d121 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x35:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302700.932693:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:23960:x1876219350276736:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 425us (747us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302700.932702:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 824 00000100:00100000:2.0:1789302700.935642:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302700.935792:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350276992 00000100:00100000:0.0:1789302700.935807:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350276992 00000100:00100000:0.0:1789302700.937213:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 168 00000100:00100000:0.0:1789302700.937233:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:23960:x1876219350276992:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:0.0:1789302700.937627:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:23960:x1876219350276992:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 400us (1998us total) trans 4294967841 rc 0/0 00000100:00100000:0.0:1789302700.937635:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 168 00000100:00100000:3.0:1789302701.126168:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302701.126316:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350277248 00000100:00100000:0.0:1789302701.126345:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350277248 00000100:00100000:0.0:1789302701.126463:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 825 00000100:00100000:0.0:1789302701.126477:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:23976:x1876219350277248:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:0.0:1789302701.126551:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302701.126580:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d128 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0197 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302701.126752:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d12f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302701.126773:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d12f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302701.126791:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d12f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302701.126806:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x35:0x0]: returning size 158 00010000:00010000:0.0:1789302701.126988:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d128 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x33:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0197 expref: 87 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302701.127010:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221f400/0x64fe43095184d12f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x35:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0197 expref: 86 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302701.127019:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x35:0x0].0x0 (ffff960f0cdf2500) 00010000:00010000:0.0:1789302701.127023:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1789302701.127027:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221f400, rc 0) 00000100:00080000:2.0:1789302701.127186:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302701.128640:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:23976:x1876219350277248:12345-192.168.203.11@tcp:101:diff.0 Request processed in 2155us (2478us total) trans 4294967842 rc 0/0 00000100:00100000:0.0:1789302701.128653:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 825 00010000:00010000:0.0:1789302701.709179:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302702.733953:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302703.759116:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302704.140857:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371097984:0@lo:400:kworker.0 00000100:00100000:2.0:1789302704.140957:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302704.141057:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371097984 00000100:00100000:3.0:1789302704.141068:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371097984 00000100:00100000:3.0:1789302704.141131:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:3.0:1789302704.141143:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371097984:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302704.146036:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371097984:12345-0@lo:400:kworker.0 Request processed in 4879us (5064us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302704.146056:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:2.0:1789302704.146264:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371097984:0@lo:400:kworker.0 00000100:00100000:2.0:1789302704.146295:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302704.146309:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371098112:0@lo:400:kworker.0 00000100:00100000:2.0:1789302704.146447:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302704.146573:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371098112 00000100:00100000:3.0:1789302704.146582:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371098112 00000100:00100000:3.0:1789302704.146641:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:3.0:1789302704.146651:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371098112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302704.146737:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371098112:12345-0@lo:400:kworker.0 Request processed in 88us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302704.146744:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1789302704.146875:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371098112:0@lo:400:kworker.0 00000100:00100000:2.0:1789302704.147384:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371098496:0@lo:400:kworker.0 00000100:00100000:2.0:1789302704.147429:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302704.147462:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302704.147468:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371098624:0@lo:400:kworker.0 00000100:00100000:2.0:1789302704.147547:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302704.147591:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371098624 00000100:00100000:2.0:1789302704.147597:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371098624 00000100:00100000:2.0:1789302704.147649:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 321 00000100:00100000:2.0:1789302704.147657:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371098624:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302704.147702:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371098624:12345-0@lo:400:kworker.0 Request processed in 46us (199us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302704.147708:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 321 00000100:00080000:2.0:1789302704.147739:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302704.147763:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371098624:0@lo:400:kworker.0 00000100:00100000:0.0:1789302704.147914:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371098496 00000100:00100000:0.0:1789302704.147924:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371098496 00000100:00100000:0.0:1789302704.147960:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:0.0:1789302704.147967:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371098496:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302704.148008:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371098496:12345-0@lo:400:kworker.0 Request processed in 42us (579us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302704.148014:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00080000:2.0:1789302704.148117:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302704.148137:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371098496:0@lo:400:kworker.0 00000100:00100000:0.0:1789302704.148564:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371098368:0@lo:400:kworker.0 00000100:00100000:0.0:1789302704.148600:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302704.148627:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302704.148633:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371098240:0@lo:400:kworker.0 00000100:00100000:0.0:1789302704.148656:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302704.148740:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371098368 00000100:00100000:3.0:1789302704.148746:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371098368 00000100:00100000:3.0:1789302704.148805:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1789302704.148813:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371098368:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302704.148859:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371098368:12345-0@lo:400:kworker.0 Request processed in 47us (257us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302704.148865:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1789302704.148884:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371098240 00000100:00100000:3.0:1789302704.148888:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371098240 00000100:00100000:3.0:1789302704.148913:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:3.0:1789302704.148919:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371098240:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302704.148950:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371098240:12345-0@lo:400:kworker.0 Request processed in 31us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302704.148956:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:3.0:1789302704.148988:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371098240:0@lo:400:kworker.0 00000100:00100000:3.0:1789302704.148997:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371098368:0@lo:400:kworker.0 00000100:00080000:0.0:1789302704.149094:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302704.149098:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302704.399263:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302704.399364:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350277376 00000100:00100000:0.0:1789302704.399378:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350277376 00000100:00100000:0.0:1789302704.399459:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 537 00000100:00100000:0.0:1789302704.399472:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350277376:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302704.399620:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350277376:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 151us (374us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302704.399631:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 537 00000100:00100000:2.0:1789302704.399974:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302704.400081:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350277504 00000100:00100000:0.0:1789302704.400087:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350277504 00000100:00100000:0.0:1789302704.400153:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:0.0:1789302704.400161:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:2348:x1876219350277504:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302704.400211:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:2348:x1876219350277504:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 50us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302704.400241:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00080000:2.0:1789302704.400653:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302704.400844:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302704.400881:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302704.400912:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350277632 00000100:00100000:1.0:1789302704.400919:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302704.400921:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350277632 00000100:00100000:1.0:1789302704.400954:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302704.400979:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350277760 00000100:00100000:2.0:1789302704.400985:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350277760 00000100:00100000:1.0:1789302704.400990:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302704.401009:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350278016 00000100:00100000:2.0:1789302704.401013:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350278016 00000100:00100000:1.0:1789302704.401019:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350278144 00000100:00100000:1.0:1789302704.401026:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350278144 00000100:00100000:2.0:1789302704.401035:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 323 00000100:00100000:2.0:1789302704.401046:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350277632:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302704.401064:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 324 00000100:00100000:1.0:1789302704.401074:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350277760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302704.401121:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350277632:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 77us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302704.401128:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 323 00000100:00100000:2.0:1789302704.401144:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:2.0:1789302704.401151:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350278016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302704.401165:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:3.0:1789302704.401180:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350278144:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302704.401237:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350277888 00000100:00100000:0.0:1789302704.401242:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350277888 00000100:00100000:0.0:1789302704.402308:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 346 00000100:00100000:0.0:1789302704.402321:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2348:x1876219350277888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302704.402385:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2348:x1876219350277888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (1466us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302704.402397:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350277760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1320us (1511us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302704.402407:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 346 00000100:00100000:1.0:1789302704.402413:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 324 00000100:00080000:1.0:1789302704.402662:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302704.402669:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302704.402681:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302704.402860:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350278016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1706us (1902us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302704.402869:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:3.0:1789302704.403523:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350278144:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 2069us (2255us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302704.403537:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:1.0:1789302704.652965:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302704.652997:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371098752:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302704.653101:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302704.653385:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371098752 00000100:00100000:2.0:1789302704.653396:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371098752 00000100:00100000:2.0:1789302704.653460:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:2.0:1789302704.653472:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371098752:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302704.653539:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 61276160 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302704.653607:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371098752:12345-0@lo:13:osp-pre-1-0.0 Request processed in 136us (505us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302704.653615:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:1.0:1789302704.653730:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371098752:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302704.780795:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302705.167419:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371098880:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302705.170624:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302705.173424:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371098880 00000100:00100000:1.0:1789302705.173451:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371098880 00000100:00100000:1.0:1789302705.173749:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 147 00000100:00100000:1.0:1789302705.173763:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371098880:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302705.173815:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 62980096 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302705.173928:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371098880:12345-0@lo:13:osp-pre-0-0.0 Request processed in 168us (3310us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302705.173935:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 147 00000100:00100000:1.0:1789302705.174025:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371098880:0@lo:13:osp-pre-0-0.0 00000001:00080000:1.0:1789302705.769171:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967842 is committed 00000001:00080000:1.0:1789302705.769215:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967837 is committed 00000020:00080000:1.0:1789302705.769232:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f1e58ae40 refcount 2 committed 0 result 0 batchid 12 00000020:00080000:1.0:1789302705.769239:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09f45b00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f1e58af00 00000020:00080000:1.0:1789302705.769243:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09f45e00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3b081400 00000020:00080000:3.0:1789302705.769360:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 12 committed batchid 11 00000020:00080000:3.0:1789302705.769913:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 11 00010000:00010000:0.0:1789302705.806091:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302705.891957:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302705.892322:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350278272 00000100:00100000:0.0:1789302705.892335:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350278272 00000100:00100000:0.0:1789302705.892410:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 826 00000100:00100000:0.0:1789302705.892423:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23903:x1876219350278272:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1789302705.894792:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:23903:x1876219350278272:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 2354us (2835us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302705.894811:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 826 00000001:00080000:1.0:1789302706.258668:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967338 is committed 00000001:00080000:1.0:1789302706.258732:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967337 is committed 00010000:00010000:3.0:1789302706.829016:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:0.0:1789302707.307645:0:18363:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12c: Restore a file with stripe of 2 == 08:31:45 (1789302705) 00000100:00100000:1.0:1789302707.595202:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302707.596797:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350278400 00000100:00100000:0.0:1789302707.596815:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350278400 00000100:00080000:1.0:1789302707.598877:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302707.598979:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 827 00000100:00100000:2.0:1789302707.598997:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:24590:x1876219350278400:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302707.599154:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:24590:x1876219350278400:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 164us (3966us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302707.599162:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 827 00000100:00100000:1.0:1789302707.615093:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302707.615248:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350278528 00000100:00100000:0.0:1789302707.615261:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350278528 00000100:00100000:0.0:1789302707.615449:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 828 00000100:00100000:0.0:1789302707.615463:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:24591:x1876219350278528:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302707.615492:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302707.615571:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d13d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a019e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.615681:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.615709:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.615726:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.615743:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.615750:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.615758:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616120:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616130:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616138:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616145:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221e400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302707.616150:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616169:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616183:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:0.0:1789302707.616187:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302707.616193:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616200:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e400/0x64fe43095184d144 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616216:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616222:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616227:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616232:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302707.616235:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616241:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616248:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff960f03501e00) 00010000:00010000:0.0:1789302707.616250:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302707.616253:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616270:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d14b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616307:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d13d lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a019e expref: 82 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616319:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:0.0:1789302707.616322:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302707.616326:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d13d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a019e expref: 82 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.616336:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221e200, rc 0) 00000100:00100000:0.0:1789302707.616812:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:24591:x1876219350278528:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1350us (1733us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302707.616823:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 828 00000100:00080000:1.0:1789302707.616951:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302707.637366:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.637685:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350278784 00000100:00100000:3.0:1789302707.637708:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350278784 00000100:00100000:3.0:1789302707.637821:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 829 00000100:00100000:3.0:1789302707.637835:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:24591:x1876219350278784:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:3.0:1789302707.637910:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302707.637945:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184cf3e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a010b expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.637968:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302707.637974:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302707.638267:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.638285:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.638297:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.638341:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 34 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.638367:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 34 pid: 5843 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302707.638657:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371099008:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302707.639091:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66c00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302707.641021:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371099008:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302707.641060:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302707.642063:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.642414:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350278912 00000100:00100000:3.0:1789302707.642425:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350278912 00010000:00010000:3.0:1789302707.642471:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 35 pid: 5843 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302707.642540:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 257 00000100:00100000:3.0:1789302707.642552:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:2348:x1876219350278912:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302707.642572:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302707.642580:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 35 pid: 5843 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.642594:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 35 pid: 5843 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.642741:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e800/0x64fe43095184cf68 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0119 expref: 35 pid: 5843 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.642760:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302707.642774:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.642784:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.642793:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.642864:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302707.642955:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2348:x1876219350278912:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 403us (895us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302707.642986:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 257 00010000:00010000:1.0:1789302707.644077:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.644092:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.644129:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.644143:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.644153:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302707.646482:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:1.0:1789302707.646600:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.646612:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.646621:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.646641:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302707.646647:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.646657:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.646678:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:1.0:1789302707.646681:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302707.646687:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.646693:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d152 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302707.646726:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008688ad3c reply state 00000000571cb778 transno 4294967843 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302707.646735:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302707.646959:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:24591:x1876219350278784:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 9122us (9612us total) trans 4294967843 rc 0/0 00000100:00100000:1.0:1789302707.646970:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 829 00010000:00010000:0.0:1789302707.651404:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.651428:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.651437:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.651444:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302707.651450:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.651459:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.651553:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff960f10d24000) 00010000:00010000:0.0:1789302707.651557:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302707.651564:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.651571:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d159 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302707.660797:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302707.660891:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350279040 00000100:00100000:0.0:1789302707.660905:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350279040 00000100:00100000:0.0:1789302707.660977:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 830 00000100:00100000:0.0:1789302707.660989:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:24592:x1876219350279040:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302707.661022:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302707.661053:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d160 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01a5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.661139:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d167 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.661156:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d167 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.661170:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d167 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302707.661214:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302707.661218:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302707.661239:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d167 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.661293:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d160 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01a5 expref: 83 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.661316:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d167 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01a5 expref: 82 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.661326:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302707.661329:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302707.661334:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221fc00, rc 0) 00000100:00100000:0.0:1789302707.661418:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:24592:x1876219350279040:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 431us (637us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302707.661425:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 830 00000100:00100000:1.0:1789302707.668448:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302707.668600:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350279168 00000100:00100000:2.0:1789302707.668613:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350279168 00000100:00100000:2.0:1789302707.668683:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 831 00000100:00100000:2.0:1789302707.668704:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:24592:x1876219350279168:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302707.668731:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302707.668757:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d16e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01ac expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.668872:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.668889:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.668904:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.668921:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.668929:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.668936:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669057:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d183 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669066:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669073:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302707.669105:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:2.0:1789302707.669109:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302707.669129:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d183 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669152:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669159:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669165:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669170:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302707.669175:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669183:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669197:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:2.0:1789302707.669199:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302707.669205:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669212:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d175 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669227:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669232:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669237:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669248:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302707.669251:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669258:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669266:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff960f03927f00) 00010000:00010000:2.0:1789302707.669269:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302707.669272:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669278:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d17c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669312:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d16e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01ac expref: 84 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669329:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d183 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01ac expref: 83 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.669337:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302707.669340:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302707.669342:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbee00, rc 0) 00000100:00100000:2.0:1789302707.669470:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:24592:x1876219350279168:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 777us (1038us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302707.669478:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 831 00000100:00100000:3.0:1789302707.673459:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302707.673610:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350279424 00000100:00100000:0.0:1789302707.673624:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350279424 00000100:00100000:0.0:1789302707.673707:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 832 00000100:00100000:0.0:1789302707.673719:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:24592:x1876219350279424:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302707.673747:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302707.673780:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d18a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01b3 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.673934:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.673952:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.673966:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.673998:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.674006:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.674013:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.674973:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.674989:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.674999:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302707.675011:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x37:0x0]: returning size 0 00010000:00010000:0.0:1789302707.675038:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675045:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675053:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675060:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221ee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302707.675065:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675087:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675101:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302707.675104:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302707.675110:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675117:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221ee00/0x64fe43095184d191 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302707.675130:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a7a17850 reply state 00000000b2a49dcb transno 4294967844 ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302707.675139:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789302707.675175:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d18a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675189:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 85 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.675197:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f03501900) 00010000:00010000:0.0:1789302707.675199:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302707.675202:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221fe00, rc 0) 00000100:00100000:0.0:1789302707.675326:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:24592:x1876219350279424:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1609us (1881us total) trans 4294967844 rc 0/0 00000100:00100000:0.0:1789302707.675335:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 832 00010000:00010000:3.0:1789302707.677722:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.677744:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.677753:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.677760:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e0221fa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302707.677765:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.677785:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.677803:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff960f03501000) 00010000:00010000:3.0:1789302707.677806:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302707.677811:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.677817:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fa00/0x64fe43095184d198 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302707.680058:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302707.680378:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350279680 00000100:00100000:2.0:1789302707.680411:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350279680 00000100:00100000:2.0:1789302707.680479:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 833 00000100:00100000:2.0:1789302707.680491:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:24592:x1876219350279680:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302707.684125:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302707.684170:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1ad lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01ba expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.684323:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.684440:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.684453:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.684518:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.684537:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 86 pid: 5845 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302707.684555:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371099136:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302707.684686:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1000 going to sleep for 11 seconds 00000100:00100000:2.0:1789302707.687809:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302707.687924:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350279808 00000100:00100000:1.0:1789302707.687936:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350279808 00010000:00010000:1.0:1789302707.687986:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 87 pid: 5845 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302707.688034:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 258 00000100:00100000:1.0:1789302707.688064:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:2347:x1876219350279808:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302707.688083:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302707.688091:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 87 pid: 5845 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.688105:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 87 pid: 5845 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.688130:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f03501900) 00010000:00010000:1.0:1789302707.688136:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.688146:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.688155:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.688177:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302707.688240:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:2347:x1876219350279808:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 178us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302707.688247:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 258 00010000:00010000:3.0:1789302707.689904:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fe00/0x64fe43095184d19f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01b3 expref: 86 pid: 5845 timeout: 644 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302707.689955:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981f80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371099136:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302707.689986:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.689996:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690003:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302707.690029:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x37:0x0]: returning size 0 00000004:00080000:3.0:1789302707.690610:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x11:0x0], index 0: 0 00000004:00080000:3.0:1789302707.690621:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x11:0x0], index 1: 0 00010000:00010000:3.0:1789302707.690794:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690804:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690813:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690819:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302707.690824:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690833:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690863:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f03501900) 00010000:00010000:3.0:1789302707.690866:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302707.690872:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690879:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1b4 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690951:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1ad lrc: 1/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01ba expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690963:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302707.690965:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302707.690969:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1ad lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01ba expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.690979:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:3.0:1789302707.691053:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:24592:x1876219350279680:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 10564us (11010us total) trans 4294967845 rc 301/301 00000100:00100000:3.0:1789302707.691064:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 833 00000100:00100000:2.0:1789302707.693527:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302707.693680:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350280064 00000100:00100000:1.0:1789302707.693691:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350280064 00000100:00100000:1.0:1789302707.693889:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 169 00000100:00100000:1.0:1789302707.693904:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:24592:x1876219350280064:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:1.0:1789302707.694876:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:24592:x1876219350280064:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 973us (1384us total) trans 4294967846 rc 0/0 00000100:00100000:1.0:1789302707.694911:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 169 00000100:00100000:3.0:1789302707.696839:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302707.696918:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350280192 00000100:00100000:0.0:1789302707.696945:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350280192 00000100:00100000:0.0:1789302707.697010:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 834 00000100:00100000:0.0:1789302707.697021:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:24592:x1876219350280192:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302707.697051:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302707.697082:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d1c2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01c1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.697159:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d1c2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01c1 expref: 87 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.697177:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d1c2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01c1 expref: 87 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.697200:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f03501300) 00010000:00010000:0.0:1789302707.697205:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302707.697209:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e0221e200, rc 0) 00000100:00100000:0.0:1789302707.697567:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:24592:x1876219350280192:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 547us (737us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302707.697575:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 834 00000100:00100000:3.0:1789302707.699883:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302707.699956:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350280448 00000100:00100000:2.0:1789302707.699968:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350280448 00000100:00100000:2.0:1789302707.700082:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 170 00000100:00100000:2.0:1789302707.700111:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:24592:x1876219350280448:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:2.0:1789302707.700431:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:24592:x1876219350280448:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 321us (558us total) trans 4294967847 rc 0/0 00000100:00100000:2.0:1789302707.700439:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 170 00000100:00100000:1.0:1789302707.739166:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.739297:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350280576 00000100:00100000:3.0:1789302707.739312:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350280576 00000100:00100000:3.0:1789302707.739433:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 835 00000100:00100000:3.0:1789302707.739445:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:24598:x1876219350280576:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302707.739476:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302707.739565:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184d1c9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01c8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.739661:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d1d0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.739680:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d1d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.739704:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d1d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302707.739750:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:3.0:1789302707.739754:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302707.739776:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d1d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.739807:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184d1c9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01c8 expref: 90 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.739829:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d1d0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01c8 expref: 89 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.739839:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302707.739842:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302707.739847:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1400, rc 0) 00000100:00100000:3.0:1789302707.739927:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:24598:x1876219350280576:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 484us (776us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302707.739935:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 835 00000100:00100000:3.0:1789302707.748862:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302707.749071:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350280704 00000100:00100000:2.0:1789302707.749084:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350280704 00000100:00100000:2.0:1789302707.749205:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 836 00000100:00100000:2.0:1789302707.749217:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:24599:x1876219350280704:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302707.749246:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302707.749275:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d1d7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01cf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749361:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749377:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749389:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749406:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749413:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749419:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749588:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1ec lrc: 3/1,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749598:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749605:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302707.749660:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x37:0x0]: returning size 0 00000004:00200000:2.0:1789302707.749666:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302707.749689:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749711:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749722:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749729:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749737:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749743:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302707.749748:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749757:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749779:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302707.749782:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302707.749789:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749796:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d1de lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749812:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749819:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749824:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749829:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302707.749832:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749838:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749846:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff960f03927f00) 00010000:00010000:2.0:1789302707.749849:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302707.749852:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749859:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d1e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749912:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d1d7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01cf expref: 91 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749929:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01cf expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.749936:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f03501300) 00010000:00010000:2.0:1789302707.749939:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302707.749942:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbec00, rc 0) 00000100:00100000:2.0:1789302707.750279:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:24599:x1876219350280704:12345-192.168.203.11@tcp:101:rm.0 Request processed in 1062us (1429us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302707.750289:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 836 00000100:00100000:1.0:1789302707.753784:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.753932:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350280832 00000100:00100000:3.0:1789302707.753945:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350280832 00000100:00100000:3.0:1789302707.754081:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 205 00000100:00100000:3.0:1789302707.754095:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350280832:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302707.754119:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302707.754146:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01d6 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302707.754181:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x11:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302707.754207:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=544s v=(0 0) 00010000:00010000:3.0:1789302707.754212:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302707.754223:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.754245:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.754270:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:3.0:1789302707.754343:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350280832:12345-192.168.203.11@tcp:101:rm.0 Request processed in 252us (581us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302707.754475:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 205 00000100:00080000:1.0:1789302707.755072:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302707.755955:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.756074:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350280960 00000100:00100000:3.0:1789302707.756085:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350280960 00000100:00100000:3.0:1789302707.756143:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 206 00000100:00100000:3.0:1789302707.756153:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350280960:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302707.756172:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302707.756210:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01dd expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302707.756238:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x11:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302707.756571:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=544s v=(0 0) 00010000:00010000:3.0:1789302707.756577:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302707.756588:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 13 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.756611:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 13 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.756626:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302707.756737:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2345:x1876219350280960:12345-192.168.203.11@tcp:101:rm.0 Request processed in 585us (786us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302707.756745:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 206 00000100:00080000:1.0:1789302707.756844:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302707.760431:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.760774:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350281216 00000100:00100000:3.0:1789302707.760786:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350281216 00000100:00100000:3.0:1789302707.760850:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 837 00000100:00100000:3.0:1789302707.760862:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:24599:x1876219350281216:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:3.0:1789302707.760933:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1789302707.760968:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d1ec lrc: 0/0,0 mode: --/PR res: [0x200000401:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01cf expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.760992:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221e200/0x64fe43095184d1c2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x37:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01c1 expref: 89 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761144:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f03501300) 00010000:00010000:3.0:1789302707.761165:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d1d0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01c8 expref: 88 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761174:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302707.761178:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302707.761206:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761222:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761233:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761247:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761254:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761274:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761409:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 3/0,1 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761418:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.761437:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302707.762424:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].9, 64 off8792 00000004:00080000:3.0:1789302707.762439:0:6919:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 544772935826ns, next at 545772932483ns, rc = 0 00000040:00080000:3.0:1789302707.762492:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].8, 64 off8776 00000004:00080000:3.0:1789302707.762499:0:6919:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 544772996755ns, next at 545772994633ns, rc = 0 00010000:00010000:3.0:1789302707.762661:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762674:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762685:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762704:0:6919:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762712:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762718:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762744:0:6919:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302707.762770:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762776:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762797:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762803:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302707.762809:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762818:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.762896:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302707.762909:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762911:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f0cdf2d00) 00010000:00010000:3.0:1789302707.762915:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789302707.762918:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302707.762921:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 2/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.762922:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT flags: 0x40010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.762929:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40018480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762930:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d20f lrc: 0/0,0 mode: --/EX res: [0x200000401:0x37:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.762940:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x37:0x0].0x0 (ffff960f0cdf2d00) 00010000:00010000:2.0:1789302707.762942:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789302707.762945:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 1/0,0 mode: --/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.762950:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d216 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x37:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762953:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762959:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762965:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762970:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302707.762972:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302707.762973:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.762974:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302707.762976:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302707.762988:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.762997:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302707.762999:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302707.763003:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.763009:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d201 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302707.763019:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006a3e2f96 reply state 00000000bd84b6cc transno 4294967848 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302707.763029:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302707.763141:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:24599:x1876219350281216:12345-192.168.203.11@tcp:36:rm.0 Request processed in 2278us (2720us total) trans 4294967848 rc 0/0 00000100:00100000:3.0:1789302707.763151:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 837 00010000:00010000:0.0:1789302707.765602:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.765625:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.765634:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.765664:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302707.765670:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.765679:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.765707:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff960f0cdf2c00) 00010000:00010000:0.0:1789302707.765711:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302707.765717:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302707.765723:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d208 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302707.771158:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302707.771490:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350281472 00000100:00100000:1.0:1789302707.771589:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350281472 00000100:00100000:1.0:1789302707.771821:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 838 00000100:00100000:1.0:1789302707.771834:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:24600:x1876219350281472:12345-192.168.203.11@tcp:101:dd.0 00010000:00010000:1.0:1789302707.771869:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302707.771898:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d21d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01e4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.772054:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.772073:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.772086:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.772103:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.772111:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.772119:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302707.772766:0:6999:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4700 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6999:1876219371099264:0@lo:5:osp-pre-0-0.0 00000100:00100000:3.0:1789302707.772843:0:6999:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302707.772907:0:6999:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1840 going to sleep for 11 seconds 00000100:00100000:2.0:1789302707.773003:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371099264 00000100:00100000:2.0:1789302707.773013:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371099264 00000100:00080000:0.0:1789302707.773076:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302707.773092:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:2.0:1789302707.773103:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:6999:x1876219371099264:12345-0@lo:5:osp-pre-0-0.0 00002000:00080000:2.0:1789302707.773151:0:6696:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0000: reserve 32 objects in group 0x240000400 at 34 00000004:00080000:1.0:1789302707.773320:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x12:0x0], index 0: 0 00010000:00010000:1.0:1789302707.774804:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.774840:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.774857:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775125:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775136:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775145:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775152:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302707.775158:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775167:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775184:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302707.775188:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302707.775194:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775214:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d224 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302707.775232:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000182cc22 reply state 0000000093905d37 transno 4294967849 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302707.775243:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302707.775634:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d21d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 91 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775657:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 90 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.775666:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302707.775669:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302707.775673:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb400, rc 0) 00000100:00100000:1.0:1789302707.775867:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:24600:x1876219350281472:12345-192.168.203.11@tcp:101:dd.0 Request processed in 4032us (4723us total) trans 4294967849 rc 0/0 00000100:00100000:1.0:1789302707.775879:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 838 00010000:00010000:1.0:1789302707.778047:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.778064:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.778073:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.778079:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302707.778096:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.778105:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.778123:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff960f10d24100) 00010000:00010000:1.0:1789302707.778127:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302707.778132:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302707.778139:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d22b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302707.782087:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.786576:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350281600 00000100:00100000:3.0:1789302707.786595:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350281600 00000100:00100000:3.0:1789302707.786675:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00100000:3.0:1789302707.786688:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:24600:x1876219350281600:12345-192.168.203.11@tcp:101:dd.0 00010000:00010000:3.0:1789302707.786723:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302707.786754:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01eb expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302707.786808:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x12:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302707.786841:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=544s v=(0 0) 00010000:00010000:3.0:1789302707.786847:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302707.786859:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 12 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.786879:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 12 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302707.786896:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302707.786978:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:24600:x1876219350281600:12345-192.168.203.11@tcp:101:dd.0 Request processed in 293us (4901us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302707.786987:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00100000:2.0:1789302707.800550:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:6999:x1876219371099264:12345-0@lo:5:osp-pre-0-0.0 Request processed in 27438us (27696us total) trans 4294967335 rc 0/0 00000100:00100000:2.0:1789302707.800568:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:3.0:1789302707.800631:0:6999:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4700 pname:cluuid:pid:xid:nid:opc:job osp-pre-0-0:lustre-MDT0000-mdtlov_UUID:6999:1876219371099264:0@lo:5:osp-pre-0-0.0 00000004:00080000:3.0:1789302707.800653:0:6999:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0000-osc-MDT0000: current precreated pool: [0x240000400:0x12:0x0]-[0x240000400:0x41:0x0] 00000100:00100000:3.0:1789302707.808096:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302707.808252:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350281728 00000100:00100000:2.0:1789302707.808265:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350281728 00002000:00100000:2.0:1789302707.808327:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff960e011c5180 x1876219350281728/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/0 e 0 to 0 dl 1789302718 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302707.808340:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c5180 with x1876219350281728 ext(0->1048575) 00002000:00010000:2.0:1789302707.808356:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c5180 00000100:00100000:2.0:1789302707.808388:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 67 00000100:00100000:2.0:1789302707.808398:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350281728:12345-192.168.203.11@tcp:4:dd.0 00000100:00100000:1.0:1789302707.837657:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2347:x1876219350281728:12345-192.168.203.11@tcp:4:dd.0 Request processed in 29244us (29561us total) trans 4294967336 rc 0/0 00000100:00100000:1.0:1789302707.837680:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 67 00002000:00100000:1.0:1789302707.837703:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff960e011c5180 x1876219350281728/t4294967336(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/448 e 0 to 0 dl 1789302718 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302707.837716:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c5180 with x1876219350281728 ext(0->1048575) 00002000:00010000:1.0:1789302707.837724:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c5180 00000100:00100000:3.0:1789302707.838616:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302707.838702:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350281856 00000100:00100000:0.0:1789302707.838713:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350281856 00002000:00100000:0.0:1789302707.838761:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (1048576->2097151) req@ffff960e011c5500 x1876219350281856/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/0 e 0 to 0 dl 1789302718 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302707.838780:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c5500 with x1876219350281856 ext(1048576->2097151) 00002000:00010000:0.0:1789302707.838784:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c5500 00000100:00100000:0.0:1789302707.838811:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 68 00000100:00100000:0.0:1789302707.838820:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350281856:12345-192.168.203.11@tcp:4:dd.0 00000100:00100000:0.0:1789302707.851889:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350281856:12345-192.168.203.11@tcp:4:dd.0 Request processed in 13055us (13262us total) trans 4294967337 rc 0/0 00000100:00100000:0.0:1789302707.851927:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 68 00002000:00100000:0.0:1789302707.851943:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (1048576->2097151) req@ffff960e011c5500 x1876219350281856/t4294967337(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/448 e 0 to 0 dl 1789302718 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302707.851956:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c5500 with x1876219350281856 ext(1048576->2097151) 00002000:00010000:0.0:1789302707.851963:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c5500 00010000:00010000:0.0:1789302707.852800:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302707.870369:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302707.870948:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350281984 00000100:00100000:2.0:1789302707.871053:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350281984 00002000:00100000:2.0:1789302707.871123:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (2097152->3145727) req@ffff960f3ef32300 x1876219350281984/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/0 e 0 to 0 dl 1789302718 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302707.871135:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef32300 with x1876219350281984 ext(2097152->3145727) 00002000:00010000:2.0:1789302707.871141:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ef32300 00000100:00100000:2.0:1789302707.871282:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 69 00000100:00100000:2.0:1789302707.871293:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350281984:12345-192.168.203.11@tcp:4:dd.0 00000100:00100000:3.0:1789302707.890023:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302707.890550:0:6701:0:(service.c:3467:ptlrpc_start_thread()) ost_io[0] started 3 min 3 max 116 00000100:00100000:0.0:1789302707.890564:0:6701:0:(service.c:3526:ptlrpc_start_thread()) starting thread 'll_ost_io00_003' 00000100:00100000:1.0:1789302707.891010:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350282112 00000100:00100000:1.0:1789302707.891023:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350282112 00002000:00100000:1.0:1789302707.891089:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (3145728->4194303) req@ffff960e011c6a00 x1876219350282112/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/0 e 0 to 0 dl 1789302718 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302707.891102:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c6a00 with x1876219350282112 ext(3145728->4194303) 00002000:00010000:1.0:1789302707.891109:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c6a00 00000100:00100000:0.0:1789302707.891597:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 70 00000100:00100000:0.0:1789302707.891611:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350282112:12345-192.168.203.11@tcp:4:dd.0 00000100:00100000:0.0:1789302707.914179:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2345:x1876219350282112:12345-192.168.203.11@tcp:4:dd.0 Request processed in 22557us (24154us total) trans 4294967338 rc 0/0 00000100:00100000:0.0:1789302707.914196:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 70 00002000:00100000:0.0:1789302707.914211:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x12:0x0] (3145728->4194303) req@ffff960e011c6a00 x1876219350282112/t4294967338(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/448 e 0 to 0 dl 1789302718 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302707.914222:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c6a00 with x1876219350282112 ext(3145728->4194303) 00002000:00010000:0.0:1789302707.914229:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c6a00 00000100:00100000:0.0:1789302707.920168:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302707.920274:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350282368 00000100:00100000:1.0:1789302707.920288:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350282368 00000100:00100000:1.0:1789302707.920359:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 171 00000100:00100000:1.0:1789302707.920371:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:24600:x1876219350282368:12345-192.168.203.11@tcp:35:dd.0 00000100:00100000:1.0:1789302707.927986:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:24600:x1876219350282368:12345-192.168.203.11@tcp:35:dd.0 Request processed in 7602us (7817us total) trans 4294967850 rc 0/0 00000100:00100000:1.0:1789302707.928003:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 171 00000100:00100000:3.0:1789302707.929068:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302707.929136:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350282496 00000100:00100000:3.0:1789302707.929147:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350282496 00002000:00100000:3.0:1789302707.929207:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (4194304->5242879) req@ffff960e011c6d80 x1876219350282496/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/0 e 0 to 0 dl 1789302718 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302707.929220:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c6d80 with x1876219350282496 ext(4194304->5242879) 00002000:00010000:3.0:1789302707.929236:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c6d80 00000100:00100000:3.0:1789302707.929265:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 71 00000100:00100000:3.0:1789302707.929277:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2345:x1876219350282496:12345-192.168.203.11@tcp:4:dd.0 00000100:00100000:0.0:1789302707.934842:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2347:x1876219350281984:12345-192.168.203.11@tcp:4:dd.0 Request processed in 63528us (64469us total) trans 4294967339 rc 0/0 00000100:00100000:0.0:1789302707.934863:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 69 00002000:00100000:0.0:1789302707.934883:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (2097152->3145727) req@ffff960f3ef32300 x1876219350281984/t4294967339(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/448 e 0 to 0 dl 1789302718 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302707.934921:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef32300 with x1876219350281984 ext(2097152->3145727) 00002000:00010000:0.0:1789302707.934929:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ef32300 00000100:00100000:0.0:1789302707.952288:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2345:x1876219350282496:12345-192.168.203.11@tcp:4:dd.0 Request processed in 22991us (23206us total) trans 4294967340 rc 0/0 00000100:00100000:0.0:1789302707.952306:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 71 00002000:00100000:0.0:1789302707.952326:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (4194304->5242879) req@ffff960e011c6d80 x1876219350282496/t4294967340(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:283/0 lens 488/448 e 0 to 0 dl 1789302718 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302707.952341:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c6d80 with x1876219350282496 ext(4194304->5242879) 00002000:00010000:0.0:1789302707.952349:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e011c6d80 00000100:00100000:0.0:1789302707.979484:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302707.981145:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350282752 00000100:00100000:1.0:1789302707.981163:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350282752 00000100:00100000:2.0:1789302707.981484:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 839 00000100:00100000:2.0:1789302707.981497:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:24603:x1876219350282752:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:2.0:1789302707.981554:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302707.981584:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d247 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01f2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.981994:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d24e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.982013:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d24e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.982027:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d24e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302707.982039:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:2.0:1789302707.982308:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d247 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f2 expref: 94 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.982332:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d24e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f2 expref: 93 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302707.982343:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302707.982407:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:2.0:1789302707.982413:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe800, rc 0) 00000100:00100000:2.0:1789302707.982797:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:24603:x1876219350282752:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 1302us (3327us total) trans 4294967851 rc 0/0 00000100:00100000:2.0:1789302707.982824:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 839 00000100:00100000:3.0:1789302708.025828:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.029523:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350282880 00000100:00100000:2.0:1789302708.029544:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350282880 00000100:00100000:2.0:1789302708.029710:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 840 00000100:00100000:2.0:1789302708.029724:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:24604:x1876219350282880:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302708.029759:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302708.029787:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d25c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a01f9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.029883:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.029900:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.029915:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302708.029961:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:2.0:1789302708.029965:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302708.029986:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.030017:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d25c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 95 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.030039:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 94 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.030074:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302708.030078:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302708.030083:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302708.030171:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:24604:x1876219350282880:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 449us (4357us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302708.030179:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 840 00000100:00100000:0.0:1789302708.034862:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.038061:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350283136 00000100:00100000:1.0:1789302708.038085:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350283136 00000100:00100000:1.0:1789302708.038185:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 841 00000100:00100000:1.0:1789302708.038359:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:24604:x1876219350283136:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:1.0:1789302708.039182:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].20, 136 off10912 00000100:00100000:1.0:1789302708.039387:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:24604:x1876219350283136:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 1190us (4538us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302708.039398:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 841 00000040:00080000:0.0:1789302708.045439:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302708.045453:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302708.045525:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 21 count 4 00000040:00080000:0.0:1789302708.045535:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302708.046140:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ae680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371099392:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302708.046402:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e5990c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302708.059207:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ae680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371099392:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302708.059788:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302708.059934:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350283264 00000100:00100000:3.0:1789302708.059944:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350283264 00000100:00100000:3.0:1789302708.060578:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 842 00000100:00100000:3.0:1789302708.060593:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:24607:x1876219350283264:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302708.060634:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302708.061830:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:24607:x1876219350283264:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 1233us (2035us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302708.061841:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 842 00000100:00100000:2.0:1789302708.065834:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.066360:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350283520 00000100:00100000:1.0:1789302708.066374:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350283520 00000100:00100000:1.0:1789302708.066453:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 843 00000100:00100000:1.0:1789302708.066465:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:24607:x1876219350283520:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302708.066492:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302708.066540:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d26a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0200 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.066953:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.067007:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.067020:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302708.067033:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:1.0:1789302708.067324:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d26a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 37 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.067347:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.067357:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302708.067360:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302708.067365:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:1.0:1789302708.067454:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:24607:x1876219350283520:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 990us (1633us total) trans 4294967852 rc 0/0 00000100:00100000:1.0:1789302708.067463:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 843 00000100:00100000:3.0:1789302708.077313:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.077413:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350283648 00000100:00100000:1.0:1789302708.077427:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350283648 00000100:00100000:1.0:1789302708.077537:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 209 00000100:00100000:1.0:1789302708.077559:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f050a4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350283648:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302708.077648:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.077673:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 21 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.077705:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x12:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302708.077710:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302708.077715:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 4/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789302708.077946:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789302708.077974:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 21 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.077993:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 21 pid: 6695 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302708.078012:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371099520:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302708.078115:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302708.084656:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371099520:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302708.084710:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.095183:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.095738:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350283776 00000100:00100000:1.0:1789302708.095753:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350283776 00010000:00010000:1.0:1789302708.095826:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 22 pid: 6695 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302708.095884:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00100000:1.0:1789302708.095896:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2347:x1876219350283776:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302708.095937:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:1.0:1789302708.095962:0:5831:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb size from disk: 0 -> 5242880 00002000:00010000:1.0:1789302708.095966:0:5831:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb mtime from disk: 0 -> 1789302707 00002000:00010000:1.0:1789302708.095969:0:5831:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x12:0x0] updating lvb ctime from disk: 0 -> 1789302707 00010000:00010000:1.0:1789302708.095977:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 22 pid: 6695 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.095995:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 22 pid: 6695 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.096029:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d240 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01eb expref: 22 pid: 6695 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.096052:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302708.096057:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302708.096064:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.096076:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.096121:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302708.096209:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b1450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2347:x1876219350283776:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 316us (1041us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302708.096218:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 259 00010000:00010000:2.0:1789302708.097113:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097128:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097163:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097174:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097183:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097191:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302708.097196:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097225:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097243:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.097251:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184d27f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302708.097340:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f050a4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2348:x1876219350283648:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 19791us (20042us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302708.097350:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 209 00000100:00100000:3.0:1789302708.110646:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.111110:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350284032 00000100:00100000:2.0:1789302708.111125:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350284032 00000100:00100000:2.0:1789302708.111197:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 844 00000100:00100000:2.0:1789302708.111209:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:24607:x1876219350284032:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302708.111368:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:24607:x1876219350284032:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 161us (736us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302708.111376:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 844 00000100:00100000:0.0:1789302708.116757:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302708.117028:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350284160 00000100:00100000:3.0:1789302708.117042:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350284160 00000100:00100000:3.0:1789302708.117117:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 210 00000100:00100000:3.0:1789302708.117131:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2346:x1876219350284160:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302708.117154:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302708.117185:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0207 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302708.117220:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x12:0x0] initial LVB size: 5242880, mtime: 0x6aa697b3, atime: 0x0, ctime: 0x6aa697b3, blocks: 0x1 00010000:00010000:3.0:1789302708.117281:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=545s v=(0 0) 00010000:00010000:3.0:1789302708.117286:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302708.117298:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 9 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.117321:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 9 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.117337:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302708.117431:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350284160:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 303us (691us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302708.117438:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 210 00000100:00100000:0.0:1789302708.122270:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.122400:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350284416 00000100:00100000:2.0:1789302708.122416:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350284416 00000100:00100000:2.0:1789302708.122491:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 845 00000100:00100000:2.0:1789302708.122524:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22331050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:24607:x1876219350284416:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302708.122671:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22331050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:24607:x1876219350284416:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 169us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302708.123477:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 845 00000100:00100000:0.0:1789302708.140093:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.140220:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350284544 00000100:00100000:1.0:1789302708.140251:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350284544 00002000:00100000:1.0:1789302708.140334:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff960f1e6aed80 x1876219350284544/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/0 e 0 to 0 dl 1789302719 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302708.140346:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6aed80 with x1876219350284544 ext(0->1048575) 00002000:00010000:1.0:1789302708.140353:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f1e6aed80 00000100:00100000:1.0:1789302708.140393:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 72 00000100:00100000:1.0:1789302708.140405:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350284544:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302708.161277:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350284544:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 20849us (21177us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789302708.161295:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 72 00002000:00100000:1.0:1789302708.161311:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x12:0x0] (0->1048575) req@ffff960f1e6aed80 x1876219350284544/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/440 e 0 to 0 dl 1789302719 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302708.161324:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6aed80 with x1876219350284544 ext(0->1048575) 00002000:00010000:1.0:1789302708.161333:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f1e6aed80 00000100:00100000:3.0:1789302708.177779:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.177962:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350284672 00000100:00100000:2.0:1789302708.177977:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350284672 00002000:00100000:2.0:1789302708.178046:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (1048576->2097151) req@ffff960f120cbb80 x1876219350284672/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/0 e 0 to 0 dl 1789302719 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302708.178067:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120cbb80 with x1876219350284672 ext(1048576->2097151) 00002000:00010000:2.0:1789302708.178074:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120cbb80 00000100:00100000:2.0:1789302708.178119:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 73 00000100:00100000:2.0:1789302708.178132:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350284672:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1789302708.198988:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350284672:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 20848us (21222us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789302708.199007:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 73 00002000:00100000:2.0:1789302708.199025:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x12:0x0] (1048576->2097151) req@ffff960f120cbb80 x1876219350284672/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/440 e 0 to 0 dl 1789302719 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302708.199036:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120cbb80 with x1876219350284672 ext(1048576->2097151) 00002000:00010000:2.0:1789302708.199045:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120cbb80 00000100:00100000:3.0:1789302708.220722:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302708.221087:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350284928 00000100:00100000:0.0:1789302708.221100:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350284928 00002000:00100000:0.0:1789302708.221165:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (3145728->4194303) req@ffff960f120c9180 x1876219350284928/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/0 e 0 to 0 dl 1789302719 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302708.221193:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c9180 with x1876219350284928 ext(3145728->4194303) 00002000:00010000:0.0:1789302708.221200:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120c9180 00000100:00100000:0.0:1789302708.221233:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 74 00000100:00100000:0.0:1789302708.221245:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350284928:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302708.225336:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.225440:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350284800 00000100:00100000:1.0:1789302708.225454:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350284800 00002000:00100000:1.0:1789302708.225540:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (2097152->3145727) req@ffff960f120c9500 x1876219350284800/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/0 e 0 to 0 dl 1789302719 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302708.225552:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c9500 with x1876219350284800 ext(2097152->3145727) 00002000:00010000:1.0:1789302708.225559:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120c9500 00000100:00100000:1.0:1789302708.225602:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 75 00000100:00100000:1.0:1789302708.225614:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350284800:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302708.251175:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350284800:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 25547us (25838us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789302708.251192:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 75 00002000:00100000:0.0:1789302708.251209:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x12:0x0] (2097152->3145727) req@ffff960f120c9500 x1876219350284800/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/440 e 0 to 0 dl 1789302719 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302708.251222:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c9500 with x1876219350284800 ext(2097152->3145727) 00002000:00010000:0.0:1789302708.251229:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120c9500 00000100:00100000:3.0:1789302708.251426:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350284928:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 30181us (30927us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789302708.251434:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 74 00002000:00100000:3.0:1789302708.251441:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x12:0x0] (3145728->4194303) req@ffff960f120c9180 x1876219350284928/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/440 e 0 to 0 dl 1789302719 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302708.251449:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c9180 with x1876219350284928 ext(3145728->4194303) 00002000:00010000:3.0:1789302708.251452:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120c9180 00000100:00100000:3.0:1789302708.286231:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.286331:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350285056 00000100:00100000:1.0:1789302708.286343:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350285056 00002000:00100000:1.0:1789302708.286430:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x12:0x0] (4194304->5242879) req@ffff960f120c8380 x1876219350285056/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/0 e 0 to 0 dl 1789302719 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302708.286443:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c8380 with x1876219350285056 ext(4194304->5242879) 00002000:00010000:1.0:1789302708.286451:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120c8380 00000100:00100000:1.0:1789302708.286488:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 76 00000100:00100000:1.0:1789302708.286532:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350285056:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1789302708.303899:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350285056:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 17382us (17664us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789302708.303919:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 76 00002000:00100000:2.0:1789302708.303938:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x12:0x0] (4194304->5242879) req@ffff960f120c8380 x1876219350285056/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:284/0 lens 488/440 e 0 to 0 dl 1789302719 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302708.303951:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c8380 with x1876219350285056 ext(4194304->5242879) 00002000:00010000:2.0:1789302708.303972:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f120c8380 00000100:00100000:3.0:1789302708.316818:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302708.316892:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350285184 00000100:00100000:0.0:1789302708.316906:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350285184 00000100:00100000:0.0:1789302708.316981:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 846 00000100:00100000:0.0:1789302708.316994:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:24607:x1876219350285184:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302708.317020:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302708.317047:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d28d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a020e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.317094:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.317112:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.317144:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.317264:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d28d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.317282:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.317293:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:0.0:1789302708.317297:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302708.317301:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2ca00, rc 0) 00000100:00100000:0.0:1789302708.317390:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24607:x1876219350285184:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 399us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302708.317398:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 846 00000100:00100000:3.0:1789302708.324713:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.325023:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350285312 00000100:00100000:2.0:1789302708.325037:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350285312 00000100:00100000:2.0:1789302708.325117:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 847 00000100:00100000:2.0:1789302708.325130:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24607:x1876219350285312:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302708.325179:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302708.325316:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24607:x1876219350285312:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 188us (626us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302708.325324:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 847 00000100:00100000:3.0:1789302708.330037:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.331785:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350285440 00000100:00100000:1.0:1789302708.331800:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350285440 00000100:00100000:1.0:1789302708.331881:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 211 00000100:00100000:1.0:1789302708.331892:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350285440:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302708.331951:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 3/1,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.331974:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302708.331979:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302708.331987:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332003:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332039:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332048:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332081:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332089:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fe00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302708.332093:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332103:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332121:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.332129:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c9fc3634/0x64fe43095184d29b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302708.332283:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350285440:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 392us (2260us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302708.332291:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 211 00000100:00100000:3.0:1789302708.335645:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.337784:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350285696 00000100:00100000:2.0:1789302708.337801:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350285696 00000100:00100000:2.0:1789302708.337908:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 848 00000100:00100000:2.0:1789302708.337920:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24607:x1876219350285696:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302708.338026:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.338060:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.338072:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 4/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.338121:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.338138:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 37 pid: 5845 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302708.338176:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371099648:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302708.338679:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302708.341636:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371099648:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302708.341677:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.343552:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302708.344436:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350285824 00000100:00100000:0.0:1789302708.344454:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350285824 00010000:00010000:0.0:1789302708.344527:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 38 pid: 5845 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302708.344573:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 260 00000100:00100000:0.0:1789302708.344585:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:2347:x1876219350285824:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302708.344937:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302708.344949:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 38 pid: 5845 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.344993:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 38 pid: 5845 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.345021:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d294 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a020e expref: 38 pid: 5845 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.345042:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:0.0:1789302708.345049:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.345061:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.345070:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302708.345098:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302708.345180:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2347:x1876219350285824:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 597us (1638us total) trans 0 rc 0/0 00010000:00010000:2.0:1789302708.345467:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345477:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345907:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345916:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345923:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345929:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e02255a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302708.345934:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345943:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345954:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302708.345966:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302708.345972:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.345978:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255a00/0x64fe43095184d2a2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302708.346226:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 260 00000040:00080000:3.0:1789302708.346303:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302708.346309:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302708.346872:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1789302708.346882:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302708.347361:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24607:x1876219350285696:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 9436us (11720us total) trans 4294967853 rc 0/0 00000100:00100000:2.0:1789302708.347375:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 848 00000100:00100000:3.0:1789302708.350209:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.350353:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350286080 00000100:00100000:2.0:1789302708.350367:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350286080 00000100:00100000:2.0:1789302708.350438:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 172 00000100:00100000:2.0:1789302708.350450:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24607:x1876219350286080:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302708.350846:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24607:x1876219350286080:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 396us (649us total) trans 4294967854 rc 0/0 00000100:00100000:2.0:1789302708.350854:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 172 00000040:00080000:1.0:1789302708.831299:0:18413:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302708.831370:0:18413:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302708.831439:0:18413:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 21 count 4 00000040:00080000:1.0:1789302708.831449:0:18413:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302708.877129:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302708.906046:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302708.907802:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350286208 00000100:00100000:2.0:1789302708.907821:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350286208 00000100:00100000:2.0:1789302708.907901:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 212 00000100:00100000:2.0:1789302708.907914:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2346:x1876219350286208:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302708.907938:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302708.907962:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0215 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.907993:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302708.907998:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302708.908008:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.908037:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.908053:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e02254600, rc 0) 00000100:00100000:2.0:1789302708.908910:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2346:x1876219350286208:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 995us (2874us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302708.908921:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 212 00000100:00100000:0.0:1789302708.930205:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.933008:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350286464 00000100:00100000:1.0:1789302708.933027:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350286464 00000100:00100000:1.0:1789302708.933110:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 173 00000100:00100000:1.0:1789302708.933122:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ecc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:24615:x1876219350286464:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:1.0:1789302708.933422:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ecc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:24615:x1876219350286464:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 301us (3231us total) trans 4294967855 rc 0/0 00000100:00100000:1.0:1789302708.933431:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 173 00000100:00100000:0.0:1789302708.936383:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302708.936460:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350286720 00000100:00100000:3.0:1789302708.936473:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350286720 00000100:00100000:3.0:1789302708.936571:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 849 00000100:00100000:3.0:1789302708.936583:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:24615:x1876219350286720:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302708.936614:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302708.936617:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789302708.936652:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d24e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f2 expref: 95 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.936673:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302708.936679:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302708.936705:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d2b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a021c expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.936891:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d2b7 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.936908:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d2b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.936922:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d2b7 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302708.936948:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:3.0:1789302708.937100:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d2b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a021c expref: 97 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.937119:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d2b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a021c expref: 96 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.937126:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302708.937129:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1789302708.937131:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe400, rc 0) 00000100:00100000:3.0:1789302708.937206:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:24615:x1876219350286720:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 625us (834us total) trans 4294967856 rc 0/0 00000100:00100000:3.0:1789302708.937213:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 849 00000100:00100000:2.0:1789302708.945459:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302708.945558:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350286848 00000100:00100000:3.0:1789302708.945571:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350286848 00000100:00100000:3.0:1789302708.945634:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 213 00000100:00100000:3.0:1789302708.945646:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2348:x1876219350286848:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:3.0:1789302708.945714:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.945740:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 8 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.945749:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.945785:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302708.945790:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302708.945796:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.945839:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.945856:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 22 pid: 6697 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.945873:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371099776:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302708.945993:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 8 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.946003:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 8 pid: 6695 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.946013:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371099904:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302708.946040:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2900 going to sleep for 11 seconds 00000100:00100000:3.0:1789302708.951439:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371099776:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302708.951459:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371099904:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302708.951540:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302708.951713:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302708.951915:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350286976 00000100:00100000:3.0:1789302708.951925:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350286976 00010000:00010000:3.0:1789302708.951970:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 23 pid: 6697 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.952016:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:3.0:1789302708.952027:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+23:2346:x1876219350286976:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302708.952045:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302708.952076:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 23 pid: 6697 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.952091:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 23 pid: 6697 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.952119:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e1a7bf44/0x64fe43095184d2a9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0215 expref: 23 pid: 6697 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.952141:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302708.952205:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2346:x1876219350286976:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 178us (492us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302708.952212:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:0.0:1789302708.973046:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302708.973287:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350287104 00000100:00100000:3.0:1789302708.973308:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350287104 00010000:00010000:3.0:1789302708.973374:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 9 pid: 6695 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.973429:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 262 00000100:00100000:3.0:1789302708.973441:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350287104:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302708.973459:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302708.973489:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 9 pid: 6695 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.974206:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 9 pid: 6695 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.974240:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184d286 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0207 expref: 9 pid: 6695 timeout: 645 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.974270:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302708.974275:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302708.974282:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.974293:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.974327:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302708.974406:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2345:x1876219350287104:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 967us (1371us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302708.974427:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 262 00010000:00010000:1.0:1789302708.974535:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974546:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974580:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974588:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974596:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974602:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302708.974607:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974628:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974642:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.974649:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184d2c5 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302708.974712:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2348:x1876219350286848:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 29069us (29267us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302708.974722:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 213 00000100:00100000:0.0:1789302708.980454:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302708.980590:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350287360 00000100:00100000:3.0:1789302708.980602:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350287360 00000100:00100000:3.0:1789302708.980664:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 174 00000100:00100000:3.0:1789302708.980675:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:24615:x1876219350287360:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:3.0:1789302708.980774:0:5847:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000401:0x38:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d2b7 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a021c expref: 96 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302708.981037:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967857 00000001:00080000:3.0:1789302708.982032:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967858 00010000:00010000:3.0:1789302708.982119:0:5847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.982138:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.982148:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 37 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.982157:0:5847:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 5/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.982203:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 37 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.982220:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 37 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.982253:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371100032:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302708.983190:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302708.983204:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 99 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302708.983217:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371100160:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302708.983243:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f050b2780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302708.993457:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371100032:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302708.993479:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cad80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371100160:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302708.993547:0:5847:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302708.996640:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302708.996677:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302708.996863:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350287488 00000100:00100000:1.0:1789302708.996877:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350287488 00010000:00010000:1.0:1789302708.996926:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 38 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302708.996968:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350287616 00000100:00100000:1.0:1789302708.996971:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350287616 00010000:00010000:1.0:1789302708.996985:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 100 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302708.997010:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:1.0:1789302708.997021:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:2347:x1876219350287488:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302708.997041:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302708.997055:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 38 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997071:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 38 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997100:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d271 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0200 expref: 38 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997115:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302708.997121:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997131:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997137:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302708.997199:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2347:x1876219350287488:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 181us (574us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302708.997206:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:1.0:1789302708.997223:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 264 00000100:00100000:1.0:1789302708.997228:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:2347:x1876219350287616:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302708.997236:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302708.997240:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 100 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997249:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 100 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997282:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d232 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01e4 expref: 100 pid: 5844 timeout: 645 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997292:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302708.997295:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997303:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997312:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302708.997338:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302708.997357:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:2347:x1876219350287616:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 128us (679us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302708.997363:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 264 00010000:00010000:2.0:1789302708.999320:0:5847:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302708.999337:0:5847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302708.999545:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967859 00000001:00080000:2.0:1789302708.999800:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967860 00000100:00100000:2.0:1789302708.999943:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371100288:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1789302709.000680:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302709.000732:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1000 going to sleep for 11 seconds 00000100:00100000:1.0:1789302709.000840:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371100288 00000100:00100000:1.0:1789302709.000849:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371100288 00000100:00100000:1.0:1789302709.000934:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 29 00000100:00100000:1.0:1789302709.000946:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5847:x1876219371100288:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1789302709.001045:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5847:x1876219371100288:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 102us (366us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.001053:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 29 00000100:00080000:2.0:1789302709.001153:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302709.001194:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371100288:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:2.0:1789302709.002097:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967861 00000100:00100000:2.0:1789302709.002191:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371100416:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1789302709.002238:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302709.002279:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.003178:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371100416 00000100:00100000:3.0:1789302709.003188:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371100416 00000100:00100000:3.0:1789302709.003248:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 30 00000100:00100000:3.0:1789302709.003259:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5847:x1876219371100416:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1789302709.003551:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+7:5847:x1876219371100416:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 293us (1311us total) trans 4294967341 rc 0/0 00000100:00100000:3.0:1789302709.003558:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 30 00000100:00100000:2.0:1789302709.004068:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371100416:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:2.0:1789302709.004151:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967862 00010000:00010000:2.0:1789302709.004196:0:5847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.004207:0:5847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.004216:0:5847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.004223:0:5847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302709.004229:0:5847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.004238:0:5847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.004256:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302709.004260:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1789302709.004265:0:5847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.004272:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184d2cc lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302709.004598:0:5847:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x39:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1789302709.004737:0:5847:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].10, 64 off8856 00000004:00080000:2.0:1789302709.004752:0:5847:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 546015249283ns, next at 547015246776ns, rc = 0 00000001:00080000:2.0:1789302709.004819:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967863 00010000:00010000:2.0:1789302709.004858:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302709.004872:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1789302709.004880:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d2b7 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a021c expref: 103 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302709.005054:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:24615:x1876219350287360:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 24380us (24614us total) trans 4294967864 rc 0/0 00000100:00100000:2.0:1789302709.005064:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 174 00000020:00080000:2.0:1789302709.005068:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:0.0:1789302709.034016:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.034074:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350287872 00000100:00100000:0.0:1789302709.034087:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350287872 00000100:00100000:0.0:1789302709.034173:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 850 00000100:00100000:0.0:1789302709.034185:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:24617:x1876219350287872:12345-192.168.203.11@tcp:101:md5sum.0 00000100:00080000:3.0:1789302709.034207:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302709.034216:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302709.034243:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184d2d3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0223 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034339:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034358:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034371:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034387:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034395:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034402:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034590:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d2e8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034599:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d2e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034619:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d2e8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302709.034632:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:0.0:1789302709.034765:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034773:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034780:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034787:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2c400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.034791:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034799:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034813:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302709.034816:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302709.034822:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034828:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d2da lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034840:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034856:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034862:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034867:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.034871:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034876:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034885:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff960f03501200) 00010000:00010000:0.0:1789302709.034888:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302709.034891:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034897:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d2e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034950:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184d2d3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0223 expref: 105 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034965:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d2e8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0223 expref: 104 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.034973:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f03501500) 00010000:00010000:0.0:1789302709.034975:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302709.034983:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d400, rc 0) 00000100:00100000:0.0:1789302709.035072:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:24617:x1876219350287872:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 889us (1072us total) trans 4294967865 rc 0/0 00000100:00100000:0.0:1789302709.035079:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 850 00000100:00100000:2.0:1789302709.039229:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.039599:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350288128 00000100:00100000:3.0:1789302709.039613:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350288128 00000100:00100000:3.0:1789302709.039868:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 34 00000100:00100000:3.0:1789302709.039885:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:24617:x1876219350288128:12345-192.168.203.11@tcp:103:md5sum.0 00010000:00010000:3.0:1789302709.039905:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302709.039944:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d2e8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0223 expref: 104 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.039977:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f03501500) 00010000:00010000:3.0:1789302709.040092:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302709.040173:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:24617:x1876219350288128:12345-192.168.203.11@tcp:103:md5sum.0 Request processed in 293us (957us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.045177:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 34 00000100:00100000:3.0:1789302709.045359:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.045403:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350288256 00000100:00100000:3.0:1789302709.045414:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350288256 00000100:00100000:3.0:1789302709.045476:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 851 00000100:00100000:3.0:1789302709.045489:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:24617:x1876219350288256:12345-192.168.203.11@tcp:58:md5sum.0 00010000:00010000:3.0:1789302709.045830:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.045852:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.045867:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302709.046247:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].21, 136 off11048 00000100:00100000:3.0:1789302709.046636:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:24617:x1876219350288256:12345-192.168.203.11@tcp:58:md5sum.0 Request processed in 1148us (1276us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.046665:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 851 00000100:00100000:3.0:1789302709.050025:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.050385:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350288384 00000100:00100000:2.0:1789302709.050400:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350288384 00000100:00100000:2.0:1789302709.050474:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 852 00000100:00100000:2.0:1789302709.050490:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:24617:x1876219350288384:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:2.0:1789302709.050532:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302709.050558:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a022a expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.050630:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.050651:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 4/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 104 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.050680:0:6919:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.050703:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 3/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 104 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.050737:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:2.0:1789302709.050741:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302709.050745:0:6919:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 3/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 104 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.050752:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e02255c00, rc 0) 00000100:00100000:2.0:1789302709.050824:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:24617:x1876219350288384:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 340us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.050833:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 852 00000100:00100000:1.0:1789302709.261014:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371100928:0@lo:400:kworker.0 00000100:00100000:1.0:1789302709.261100:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302709.261174:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302709.261197:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371100800:0@lo:400:kworker.0 00000100:00100000:1.0:1789302709.261221:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302709.261311:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371100672:0@lo:400:kworker.0 00000100:00100000:3.0:1789302709.261344:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302709.261371:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789302709.261377:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371100544:0@lo:400:kworker.0 00000100:00100000:3.0:1789302709.261408:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302709.261433:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371101056:0@lo:400:kworker.0 00000100:00100000:3.0:1789302709.261458:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302709.261490:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371100544 00000100:00100000:3.0:1789302709.261498:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371100544 00000100:00100000:3.0:1789302709.261576:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 538 00000100:00100000:3.0:1789302709.261586:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371100544:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302709.261826:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371100928 00000100:00100000:2.0:1789302709.261831:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371100928 00000100:00100000:2.0:1789302709.261871:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371100800 00000100:00100000:2.0:1789302709.261874:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371100800 00000100:00100000:2.0:1789302709.261887:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371100672 00000100:00100000:2.0:1789302709.261889:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371100672 00000100:00100000:2.0:1789302709.261901:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 347 00000100:00100000:2.0:1789302709.261907:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371100928:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302709.261975:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371101056 00000100:00100000:1.0:1789302709.261979:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371101056 00000100:00100000:1.0:1789302709.262011:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 327 00000100:00100000:1.0:1789302709.262020:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371101056:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302709.262061:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371100544:12345-0@lo:400:kworker.0 Request processed in 476us (650us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.262069:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 538 00000100:00080000:3.0:1789302709.262110:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302709.262116:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302709.262142:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371100544:0@lo:400:kworker.0 00000100:00100000:3.0:1789302709.262177:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1789302709.262184:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371100800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302709.262288:0:5845:0:(service.c:3467:ptlrpc_start_thread()) mdt[0] started 4 min 3 max 116 00000100:00100000:0.0:1789302709.262300:0:5845:0:(service.c:3526:ptlrpc_start_thread()) starting thread 'mdt00_004' 00000100:00100000:0.0:1789302709.262439:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:0.0:1789302709.262447:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371100672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302709.263818:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371100928:12345-0@lo:400:kworker.0 Request processed in 1902us (2708us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.263831:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 347 00000100:00100000:1.0:1789302709.264044:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371101056:12345-0@lo:400:kworker.0 Request processed in 2024us (2583us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.264056:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 327 00000100:00100000:1.0:1789302709.264114:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967338 gen 2 00000004:00080000:1.0:1789302709.264122:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967338 committed transno 4294967338 00000100:00100000:1.0:1789302709.264163:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef30000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371101056:0@lo:400:kworker.0 00000100:00100000:1.0:1789302709.264195:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef31180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371100928:0@lo:400:kworker.0 00000100:00100000:3.0:1789302709.264312:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371100800:12345-0@lo:400:kworker.0 Request processed in 2126us (3088us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.264321:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:0.0:1789302709.264479:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371100672:12345-0@lo:400:kworker.0 Request processed in 2031us (3131us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.264487:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00100000:1.0:1789302709.264572:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371100800:0@lo:400:kworker.0 00000100:00100000:3.0:1789302709.264730:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371100672:0@lo:400:kworker.0 00000040:00080000:0.0:1789302709.389584:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302709.389598:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302709.389793:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 22 count 4 00000040:00080000:0.0:1789302709.389802:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302709.389992:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371101184:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302709.390141:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599a80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302709.395734:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371101184:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302709.400542:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.401172:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350288512 00000100:00100000:2.0:1789302709.401186:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350288512 00000100:00100000:2.0:1789302709.401291:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 853 00000100:00100000:2.0:1789302709.401304:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24618:x1876219350288512:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302709.401359:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302709.401541:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24618:x1876219350288512:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 240us (1043us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.401568:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 853 00000100:00080000:3.0:1789302709.401708:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302709.408471:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.408894:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350288640 00000100:00100000:2.0:1789302709.408908:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350288640 00000100:00100000:2.0:1789302709.408982:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 854 00000100:00100000:2.0:1789302709.408994:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24618:x1876219350288640:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1789302709.409200:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409222:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409237:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409253:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409269:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409275:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409301:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409309:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.409315:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302709.410079:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00000004:00200000:2.0:1789302709.410084:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302709.410101:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410112:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410118:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410123:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410127:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e02255800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302709.410130:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410136:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410147:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302709.410150:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302709.410154:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410158:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d304 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410170:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410186:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410190:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410193:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e02254200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302709.410194:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410198:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410202:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xb1925f2 (ffff960f03927f00) 00010000:00010000:2.0:1789302709.410203:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302709.410205:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410208:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d30b lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xb1925f2 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410220:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410224:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410228:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410230:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e02254800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302709.410235:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410240:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410244:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:2.0:1789302709.410245:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302709.410247:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.410250:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d312 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302709.410407:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24618:x1876219350288640:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 1413us (1949us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.410416:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 854 00000100:00100000:3.0:1789302709.422069:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.422218:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350288896 00000100:00100000:0.0:1789302709.422231:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350288896 00000100:00100000:0.0:1789302709.422476:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 855 00000100:00100000:0.0:1789302709.422488:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:24618:x1876219350288896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.422533:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302709.422562:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184d319 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0231 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423078:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423111:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423127:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302709.423142:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00010000:00010000:0.0:1789302709.423264:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423273:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423280:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423286:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2cc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.423291:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423301:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423315:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:0.0:1789302709.423319:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302709.423324:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423331:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2cc00/0x64fe43095184d320 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423527:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184d319 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0231 expref: 39 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423539:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302709.423542:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302709.423546:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2de00/0x64fe43095184d319 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0231 expref: 39 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.423557:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2de00, rc 0) 00000100:00100000:0.0:1789302709.423718:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350288896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1231us (1663us total) trans 4294967866 rc 301/301 00000100:00100000:0.0:1789302709.423727:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 855 00000100:00100000:1.0:1789302709.427590:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.427709:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350289024 00000100:00100000:0.0:1789302709.427723:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350289024 00000100:00100000:0.0:1789302709.427792:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 856 00000100:00100000:0.0:1789302709.427803:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350289024:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302709.427858:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x38:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302709.427996:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350289024:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 194us (420us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.428003:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 856 00000100:00100000:0.0:1789302709.430607:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.431065:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350289152 00000100:00100000:3.0:1789302709.431079:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350289152 00000100:00100000:3.0:1789302709.431152:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 857 00000100:00100000:3.0:1789302709.431164:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350289152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302709.431192:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302709.431220:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d32e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0238 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.431321:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.431345:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.431360:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302709.431407:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302709.431411:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302709.431434:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.431473:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d32e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 40 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.431550:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 39 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.431560:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:3.0:1789302709.431564:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302709.431568:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:3.0:1789302709.431633:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:24618:x1876219350289152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 472us (1039us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.431641:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 857 00000100:00100000:1.0:1789302709.435535:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.435726:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350289280 00000100:00100000:0.0:1789302709.435740:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350289280 00000100:00100000:0.0:1789302709.435824:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 858 00000100:00100000:0.0:1789302709.435955:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:24618:x1876219350289280:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.436004:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302709.436032:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d33c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a023f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436119:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436138:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436154:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436173:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436181:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436194:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436318:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436328:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436336:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302709.436371:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:0.0:1789302709.436382:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302709.436409:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436425:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436433:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436442:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436970:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2ca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.436978:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.436989:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437006:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302709.437010:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302709.437016:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437024:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2ca00/0x64fe43095184d343 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437041:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437048:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437069:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437076:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e03c2d400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.437079:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437087:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437095:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff960f03501200) 00010000:00010000:0.0:1789302709.437098:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302709.437102:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437108:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d400/0x64fe43095184d34a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437154:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c400/0x64fe43095184d33c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 41 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437171:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.437179:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302709.437182:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302709.437185:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e03c2d600, rc 0) 00000100:00100000:0.0:1789302709.437323:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:24618:x1876219350289280:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1487us (1827us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.437342:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 858 00000100:00100000:1.0:1789302709.441964:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.442076:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350289536 00000100:00100000:3.0:1789302709.442089:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350289536 00000100:00100000:3.0:1789302709.442208:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 859 00000100:00100000:3.0:1789302709.442221:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:24618:x1876219350289536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302709.442250:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302709.442276:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d358 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0246 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.443233:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184d35f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.443254:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184d35f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.443268:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184d35f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302709.443281:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 0 00010000:00010000:3.0:1789302709.443356:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d358 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0246 expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.443376:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184d35f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0246 expref: 42 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.443386:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f0cdf2d00) 00010000:00010000:3.0:1789302709.443389:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302709.443393:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:3.0:1789302709.443564:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:24618:x1876219350289536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1344us (1613us total) trans 4294967867 rc 0/0 00000100:00100000:3.0:1789302709.443574:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 859 00000100:00100000:0.0:1789302709.445981:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.446101:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350289792 00000100:00100000:2.0:1789302709.446113:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350289792 00000100:00100000:2.0:1789302709.446176:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 860 00000100:00100000:2.0:1789302709.446188:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:24618:x1876219350289792:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302709.446259:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302709.446291:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184d35f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0246 expref: 42 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.446312:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f0cdf2d00) 00010000:00010000:2.0:1789302709.446324:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302709.446361:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.446400:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 103 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.446410:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 4/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.446448:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 103 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.446464:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 103 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302709.446481:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371101312:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302709.446635:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d10c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302709.448834:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371101312:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302709.448866:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.449046:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.449097:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350289920 00000100:00100000:2.0:1789302709.449106:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350289920 00010000:00010000:2.0:1789302709.449149:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 104 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302709.449189:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 265 00000100:00100000:2.0:1789302709.449199:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2348:x1876219350289920:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302709.449217:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302709.449237:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 104 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.449250:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 104 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.449284:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d263 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01f9 expref: 104 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.449302:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302709.449306:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.449316:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.449325:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.449350:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302709.449403:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b0050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2348:x1876219350289920:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 206us (359us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.449411:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 265 00010000:00010000:0.0:1789302709.449452:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449462:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449488:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449498:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449526:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449609:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449618:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449624:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449630:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e02254800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.449636:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449655:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449665:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302709.449668:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302709.449672:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449678:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254800/0x64fe43095184d36d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449700:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449707:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449713:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449719:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43e600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.449722:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449728:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449735:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x15791de1 (ffff960f03501200) 00010000:00010000:0.0:1789302709.449737:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302709.449740:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.449756:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e600/0x64fe43095184d374 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x15791de1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.449817:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:24618:x1876219350289792:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 3631us (3848us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302709.449825:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 860 00000100:00100000:1.0:1789302709.451851:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.451967:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350290048 00000100:00100000:0.0:1789302709.451978:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350290048 00000100:00100000:0.0:1789302709.452046:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 861 00000100:00100000:0.0:1789302709.452058:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:24618:x1876219350290048:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.452082:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302709.452108:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e400/0x64fe43095184d37b lrc: 2/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a024d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.452148:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.452168:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.452180:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.452301:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e400/0x64fe43095184d37b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.452327:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.452445:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:0.0:1789302709.452449:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302709.452454:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f1d43e000, rc 0) 00000100:00100000:0.0:1789302709.452763:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:24618:x1876219350290048:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 700us (915us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.452771:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 861 00000100:00100000:3.0:1789302709.455627:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.455753:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350290304 00000100:00100000:0.0:1789302709.455767:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350290304 00000100:00100000:0.0:1789302709.455878:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 862 00000100:00100000:0.0:1789302709.455891:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:24618:x1876219350290304:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.456002:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.456024:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.456037:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302709.456290:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000535fe0f0 reply state 0000000053ec23f8 transno 4294967868 ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.456301:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302709.456426:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:24618:x1876219350290304:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 537us (1157us total) trans 4294967868 rc 0/0 00000100:00100000:0.0:1789302709.456434:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 862 00010000:00010000:0.0:1789302709.457807:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.457840:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.457849:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.457856:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.457862:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.457872:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.457892:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:0.0:1789302709.457895:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:0.0:1789302709.457902:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.457908:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43fc00/0x64fe43095184d389 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302709.458303:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.458369:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350290432 00000100:00100000:0.0:1789302709.458380:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350290432 00000100:00100000:0.0:1789302709.458434:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 863 00000100:00100000:0.0:1789302709.458444:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:24618:x1876219350290432:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.458469:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302709.458488:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0254 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.458588:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 44 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.458603:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 44 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.458612:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:0.0:1789302709.458615:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302709.458618:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f1d43f800, rc 0) 00000100:00100000:0.0:1789302709.458676:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:24618:x1876219350290432:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 234us (376us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.458696:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 863 00000100:00100000:2.0:1789302709.460312:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.460391:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350290560 00000100:00100000:0.0:1789302709.460403:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350290560 00000100:00100000:0.0:1789302709.460538:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 175 00000100:00100000:0.0:1789302709.460550:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:24618:x1876219350290560:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:0.0:1789302709.460678:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:24618:x1876219350290560:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 130us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.460699:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 175 00000100:00100000:1.0:1789302709.462647:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.462841:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350290816 00000100:00100000:2.0:1789302709.462852:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350290816 00000100:00100000:2.0:1789302709.462921:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 176 00000100:00100000:2.0:1789302709.462932:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:24618:x1876219350290816:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302709.463634:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:24618:x1876219350290816:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 693us (995us total) trans 4294967869 rc 0/0 00000100:00100000:2.0:1789302709.463660:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 176 00000100:00100000:3.0:1789302709.465802:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.466231:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350291072 00000100:00100000:0.0:1789302709.466244:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350291072 00000100:00100000:0.0:1789302709.466311:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 864 00000100:00100000:0.0:1789302709.466322:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:24618:x1876219350291072:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302709.466474:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:24618:x1876219350291072:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 153us (682us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.466495:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 864 00000100:00100000:0.0:1789302709.474346:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.474447:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350291328 00000100:00100000:3.0:1789302709.474461:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350291328 00000100:00100000:3.0:1789302709.474563:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 865 00000100:00100000:3.0:1789302709.474575:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:24618:x1876219350291328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302709.474612:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302709.474649:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d397 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a025b expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.474797:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.474816:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.474848:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 4/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.474889:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.474905:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 46 pid: 5844 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302709.474922:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371101440:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302709.475338:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f050b2840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.488665:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371101440:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302709.488722:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.489236:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.493008:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350291456 00000100:00100000:3.0:1789302709.493026:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350291456 00010000:00010000:3.0:1789302709.493087:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 47 pid: 5844 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302709.493142:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 266 00000100:00100000:3.0:1789302709.493154:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2345:x1876219350291456:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302709.493174:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302709.493181:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 47 pid: 5844 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.493217:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 47 pid: 5844 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.493244:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f800/0x64fe43095184d390 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0254 expref: 47 pid: 5844 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.493278:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:3.0:1789302709.493285:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.493297:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.493306:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.493462:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302709.493594:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2345:x1876219350291456:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 442us (4360us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.493604:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 266 00010000:00010000:0.0:1789302709.493869:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.493879:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302709.493905:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 0 00000100:00100000:2.0:1789302709.494133:0:8001:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job osp-pre-1-0:lustre-MDT0000-mdtlov_UUID:8001:1876219371101568:0@lo:5:osp-pre-1-0.0 00000004:00080000:0.0:1789302709.494171:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x12:0x0], index 1: 0 00000100:00100000:2.0:1789302709.494219:0:8001:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302709.494273:0:8001:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d16c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.494328:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371101568 00000100:00100000:3.0:1789302709.494335:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371101568 00000100:00100000:3.0:1789302709.494381:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:3.0:1789302709.494389:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:8001:x1876219371101568:12345-0@lo:5:osp-pre-1-0.0 00002000:00080000:3.0:1789302709.494452:0:6697:0:(ofd_dev.c:1747:ofd_create_hdl()) lustre-OST0001: reserve 32 objects in group 0x280000400 at 34 00010000:00010000:0.0:1789302709.494538:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.494548:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.494557:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.494563:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.494568:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.494577:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.494590:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:0.0:1789302709.494593:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302709.494598:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.494622:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d39e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.495199:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d397 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a025b expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.495211:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302709.495214:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302709.495218:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d397 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a025b expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.495229:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:0.0:1789302709.495532:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:24618:x1876219350291328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 20926us (21176us total) trans 4294967870 rc 301/301 00000100:00100000:0.0:1789302709.495544:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 865 00000100:00100000:1.0:1789302709.500637:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.500735:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350291712 00000100:00100000:2.0:1789302709.500749:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350291712 00000100:00100000:2.0:1789302709.500814:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:2.0:1789302709.500828:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:24618:x1876219350291712:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302709.501256:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:24618:x1876219350291712:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 429us (628us total) trans 4294967871 rc 0/0 00000100:00100000:2.0:1789302709.501266:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 177 00000100:00100000:3.0:1789302709.510094:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:8001:x1876219371101568:12345-0@lo:5:osp-pre-1-0.0 Request processed in 15694us (15864us total) trans 4294967339 rc 0/0 00000100:00100000:3.0:1789302709.510122:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1789302709.510125:0:8001:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981500 pname:cluuid:pid:xid:nid:opc:job osp-pre-1-0:lustre-MDT0000-mdtlov_UUID:8001:1876219371101568:0@lo:5:osp-pre-1-0.0 00000004:00080000:2.0:1789302709.510146:0:8001:0:(osp_precreate.c:705:osp_precreate_send()) lustre-OST0001-osc-MDT0000: current precreated pool: [0x280000400:0x12:0x0]-[0x280000400:0x41:0x0] 00000100:00100000:1.0:1789302709.512545:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.512641:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350291840 00000100:00100000:2.0:1789302709.512653:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350291840 00000100:00100000:2.0:1789302709.512735:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 866 00000100:00100000:2.0:1789302709.512748:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:24618:x1876219350291840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302709.512777:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302709.512804:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d3ac lrc: 2/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0262 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.512895:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255400/0x64fe43095184d3b3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.512911:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255400/0x64fe43095184d3b3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.512939:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255400/0x64fe43095184d3b3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302709.512986:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x16:0x0]: returning size 0 00000004:00200000:2.0:1789302709.512992:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302709.513015:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255400/0x64fe43095184d3b3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.513045:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255800/0x64fe43095184d3ac lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0262 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.513064:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255400/0x64fe43095184d3b3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0262 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.513073:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:2.0:1789302709.513076:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1789302709.513080:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e02255400, rc 0) 00000100:00100000:2.0:1789302709.513151:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:24618:x1876219350291840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 406us (652us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.513159:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 866 00000100:00100000:1.0:1789302709.515866:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.515919:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.515953:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.515986:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.516004:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350292096 00000100:00100000:2.0:1789302709.516017:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350292096 00000100:00100000:1.0:1789302709.516035:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350292224 00000100:00100000:1.0:1789302709.516041:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350292224 00000100:00100000:1.0:1789302709.516167:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350292352 00000100:00100000:1.0:1789302709.516172:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350292352 00000100:00100000:3.0:1789302709.516176:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 328 00000100:00100000:2.0:1789302709.516187:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:3.0:1789302709.516192:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350292224:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302709.516202:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350292096:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302709.516217:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350291968 00000100:00100000:0.0:1789302709.516222:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350291968 00000100:00100000:0.0:1789302709.516258:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 539 00000100:00100000:0.0:1789302709.516266:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350291968:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302709.516282:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350292096:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 84us (427us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.516300:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350291968:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.516308:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 539 00000100:00100000:2.0:1789302709.516376:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:1.0:1789302709.516454:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 215 00000100:00100000:1.0:1789302709.516466:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350292352:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00000100:00080000:2.0:1789302709.516470:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302709.516478:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350292224:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 292us (524us total) trans 0 rc 0/0 00010000:00010000:1.0:1789302709.516479:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1789302709.516485:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 328 00010000:00010000:1.0:1789302709.516540:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 2/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0269 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302709.516575:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x12:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789302709.516596:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=546s v=(0 0) 00010000:00010000:1.0:1789302709.516601:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302709.516612:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.516632:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.516648:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7fe00, rc 0) 00000100:00100000:1.0:1789302709.516670:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350292352:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 208us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.516676:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 215 00000100:00100000:1.0:1789302709.519360:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.519630:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350292608 00000100:00100000:2.0:1789302709.519643:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350292608 00000100:00100000:2.0:1789302709.519951:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 867 00000100:00100000:2.0:1789302709.519966:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:24618:x1876219350292608:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302709.520114:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:24618:x1876219350292608:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 152us (766us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.520123:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 867 00000100:00100000:1.0:1789302709.527774:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.527915:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350292736 00000100:00100000:2.0:1789302709.527997:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350292736 00000100:00100000:0.0:1789302709.528434:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 216 00000100:00100000:0.0:1789302709.528462:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:24618:x1876219350292736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.528487:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302709.528664:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0270 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.528782:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.528793:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302709.528798:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302709.528804:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.528910:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.528928:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 11 pid: 6696 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.528946:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371101696:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302709.529077:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e5996c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302709.538771:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.538925:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350292864 00000100:00100000:3.0:1789302709.539047:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350292864 00010000:00010000:3.0:1789302709.539109:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 12 pid: 6696 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302709.539290:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 267 00000100:00100000:3.0:1789302709.539302:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350292864:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302709.539321:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302709.539357:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 12 pid: 6696 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.539476:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ae680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371101696:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302709.539558:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 5 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.539581:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f1d43fc00, rc 0) 00010000:00010000:3.0:1789302709.539812:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 12 pid: 6696 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.539904:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c9fc3634/0x64fe43095184d3ba lrc: 0/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0269 expref: 12 pid: 6696 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.539921:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00000100:00100000:0.0:1789302709.539971:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:24618:x1876219350292736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 11525us (12211us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.539998:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 216 00010000:00010000:3.0:1789302709.540216:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302709.540227:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.540238:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.545005:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.545440:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ae680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371101824:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302709.545601:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e5996c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302709.556595:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ae680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371101824:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302709.556633:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302709.557273:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350292864:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 17969us (18523us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.557288:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 267 00000100:00100000:1.0:1789302709.566692:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.567113:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350292992 00000100:00100000:0.0:1789302709.567129:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350292992 00002000:00100000:0.0:1789302709.567200:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x12:0x0] (0->1048575) req@ffff960f25fb9500 x1876219350292992/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302709.567214:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fb9500 with x1876219350292992 ext(0->1048575) 00002000:00010000:0.0:1789302709.567222:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25fb9500 00000100:00100000:0.0:1789302709.567320:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 77 00000100:00100000:0.0:1789302709.567344:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350292992:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1789302709.581994:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.582091:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350293120 00000100:00100000:2.0:1789302709.582103:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350293120 00002000:00100000:2.0:1789302709.582159:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x12:0x0] (1048576->2097151) req@ffff960f25fbad80 x1876219350293120/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302709.582168:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fbad80 with x1876219350293120 ext(1048576->2097151) 00002000:00010000:2.0:1789302709.582173:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25fbad80 00000100:00100000:2.0:1789302709.582203:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 78 00000100:00100000:2.0:1789302709.582211:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350293120:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1789302709.589029:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2348:x1876219350292992:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 21677us (22341us total) trans 4294967340 rc 0/0 00000100:00100000:0.0:1789302709.589049:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 77 00002000:00100000:0.0:1789302709.589067:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x12:0x0] (0->1048575) req@ffff960f25fb9500 x1876219350292992/t4294967340(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/448 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302709.589100:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fb9500 with x1876219350292992 ext(0->1048575) 00002000:00010000:0.0:1789302709.589108:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25fb9500 00000100:00100000:0.0:1789302709.591014:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.591086:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350293248 00000100:00100000:3.0:1789302709.591098:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350293248 00002000:00100000:3.0:1789302709.591154:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x12:0x0] (2097152->3145727) req@ffff960f1e6ae680 x1876219350293248/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302709.591165:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6ae680 with x1876219350293248 ext(2097152->3145727) 00002000:00010000:3.0:1789302709.591171:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1e6ae680 00000100:00100000:3.0:1789302709.591218:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 79 00000100:00100000:3.0:1789302709.591230:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2345:x1876219350293248:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302709.593740:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2348:x1876219350293120:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 11518us (11746us total) trans 4294967341 rc 0/0 00000100:00100000:2.0:1789302709.593802:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 78 00002000:00100000:2.0:1789302709.593814:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x12:0x0] (1048576->2097151) req@ffff960f25fbad80 x1876219350293120/t4294967341(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/448 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302709.593823:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fbad80 with x1876219350293120 ext(1048576->2097151) 00002000:00010000:2.0:1789302709.593829:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25fbad80 00000100:00100000:0.0:1789302709.596812:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ae680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2345:x1876219350293248:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 5573us (5799us total) trans 4294967342 rc 0/0 00000100:00100000:0.0:1789302709.596824:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 79 00002000:00100000:0.0:1789302709.596836:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x12:0x0] (2097152->3145727) req@ffff960f1e6ae680 x1876219350293248/t4294967342(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/448 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302709.596844:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6ae680 with x1876219350293248 ext(2097152->3145727) 00002000:00010000:0.0:1789302709.596850:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1e6ae680 00000100:00100000:0.0:1789302709.608026:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.608204:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350293376 00000100:00100000:1.0:1789302709.608218:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350293376 00002000:00100000:1.0:1789302709.608290:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x12:0x0] (3145728->4194303) req@ffff960f1e6ad880 x1876219350293376/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.608303:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6ad880 with x1876219350293376 ext(3145728->4194303) 00002000:00010000:1.0:1789302709.608311:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1e6ad880 00000100:00100000:1.0:1789302709.608349:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 80 00000100:00100000:1.0:1789302709.608361:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2346:x1876219350293376:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302709.622540:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.623637:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350293504 00000100:00100000:3.0:1789302709.623651:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350293504 00000100:00100000:1.0:1789302709.623940:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2346:x1876219350293376:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 15576us (15926us total) trans 4294967343 rc 0/0 00000100:00100000:1.0:1789302709.623953:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 80 00002000:00100000:1.0:1789302709.623968:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x12:0x0] (3145728->4194303) req@ffff960f1e6ad880 x1876219350293376/t4294967343(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/448 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.623979:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6ad880 with x1876219350293376 ext(3145728->4194303) 00002000:00010000:1.0:1789302709.623986:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1e6ad880 00002000:00100000:2.0:1789302709.624596:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x12:0x0] (4194304->5242879) req@ffff960e03983b80 x1876219350293504/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302709.624608:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03983b80 with x1876219350293504 ext(4194304->5242879) 00002000:00010000:2.0:1789302709.624613:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03983b80 00000100:00100000:2.0:1789302709.624651:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 81 00000100:00100000:2.0:1789302709.624662:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350293504:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302709.636892:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350293504:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 12216us (14389us total) trans 4294967344 rc 0/0 00000100:00100000:2.0:1789302709.636909:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 81 00002000:00100000:2.0:1789302709.636937:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x12:0x0] (4194304->5242879) req@ffff960e03983b80 x1876219350293504/t4294967344(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:285/0 lens 488/448 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302709.636950:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03983b80 with x1876219350293504 ext(4194304->5242879) 00002000:00010000:2.0:1789302709.636958:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03983b80 00000100:00100000:3.0:1789302709.642068:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.642238:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350293632 00000100:00100000:0.0:1789302709.642256:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350293632 00000100:00100000:0.0:1789302709.642338:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 868 00000100:00100000:0.0:1789302709.642352:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:24618:x1876219350293632:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:0.0:1789302709.692008:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967870 is committed 00000001:00080000:0.0:1789302709.692034:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967865 is committed 00000001:00080000:0.0:1789302709.692105:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967871 is committed 00000004:00080000:0.0:1789302709.692117:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 11 committed 00000020:00080000:0.0:1789302709.692136:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f103d1000 refcount 2 committed 0 result 0 batchid 13 00000020:00080000:0.0:1789302709.692143:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09f45400 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f103d1480 00000020:00080000:0.0:1789302709.692146:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09f45100 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f09672400 00000020:00080000:0.0:1789302709.692336:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 13 committed batchid 11 00000020:00080000:0.0:1789302709.692572:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 11 00000100:00100000:0.0:1789302709.692594:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371101952:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789302709.692935:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302709.693047:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371102080:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789302709.693077:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1789302709.693141:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 9 committed 00000004:00080000:0.0:1789302709.693155:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 10 committed 00000020:00080000:0.0:1789302709.693168:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 12 batchid updated 00000100:00100000:0.0:1789302709.693254:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371102208:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789302709.693286:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302709.693328:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371101952 00000100:00100000:0.0:1789302709.693338:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371101952 00000100:00100000:0.0:1789302709.693387:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371102080 00000100:00100000:0.0:1789302709.693391:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371102080 00000100:00100000:0.0:1789302709.693407:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371102208 00000100:00100000:0.0:1789302709.693409:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371102208 00000100:00100000:0.0:1789302709.693426:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:0.0:1789302709.693436:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+8:3303:x1876219371101952:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789302709.693467:0:13439:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:17 count 1 00010000:00010000:0.0:1789302709.693527:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.693554:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 20 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.693564:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=2s v=(0 0) 00010000:00010000:0.0:1789302709.693570:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302709.693575:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.693618:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 20 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.693635:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 20 pid: 6695 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.693647:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371102336:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302709.693877:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789302709.693888:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3300:x1876219371102080:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789302709.693904:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:18 count 1 00002000:00010000:2.0:1789302709.693932:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x12:0x0] initial LVB size: 5242880, mtime: 0x6aa697b3, atime: 0x0, ctime: 0x6aa697b3, blocks: 0x1 00010000:00010000:2.0:1789302709.693940:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 3/0,1 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.693951:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=546s v=(0 0) 00010000:00010000:2.0:1789302709.693954:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302709.693960:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.693985:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.693997:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694004:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694024:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789302709.694039:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302709.694045:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694053:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e02255600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302709.694058:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694069:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694086:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 1/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694094:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004ca29c5e/0x64fe43095184d3cf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694121:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302709.694123:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302709.694125:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302709.694488:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3300:x1876219371102080:12345-0@lo:6:osp-syn-0-0.0 Request processed in 601us (1409us total) trans 4294967342 rc 0/0 00000100:00100000:2.0:1789302709.694496:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1789302709.694556:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:2.0:1789302709.694564:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371102208:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1789302709.694579:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:17 count 1 00010000:00010000:2.0:1789302709.694595:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694609:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694616:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=2s v=(0 0) 00010000:00010000:2.0:1789302709.694619:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302709.694623:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694660:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.694671:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 12 pid: 6697 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302709.694708:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371102464:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302709.694884:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d16c0 going to sleep for 11 seconds 00000004:00080000:1.0:1789302709.695547:0:3300:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f1e6af100/2, rc 0, transno 4294967342 00000100:00100000:1.0:1789302709.695556:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371102080:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789302709.695895:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e5996c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.697257:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350293632:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 54898us (55213us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.697271:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 868 00000100:00100000:2.0:1789302709.698398:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.698431:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.698462:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350293760 00000100:00100000:2.0:1789302709.698472:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350293760 00010000:00010000:2.0:1789302709.698539:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 13 pid: 6697 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302709.698572:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350293888 00000100:00100000:2.0:1789302709.698575:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350293888 00010000:00010000:2.0:1789302709.698605:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 21 pid: 6695 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302709.698627:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 268 00000100:00100000:2.0:1789302709.698638:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350293760:12345-192.168.203.11@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1789302709.698792:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371102336:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302709.698826:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.698993:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 269 00000100:00100000:1.0:1789302709.699005:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2347:x1876219350293888:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302709.699018:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302709.699025:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 21 pid: 6695 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.699062:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 21 pid: 6695 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.699089:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184d1f3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01d6 expref: 21 pid: 6695 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.699107:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302709.699111:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302709.699116:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.699126:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.699160:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302709.699219:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2347:x1876219350293888:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 217us (786us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.699227:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 269 00010000:00010000:0.0:1789302709.699325:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699336:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699348:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699357:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699386:0:13439:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302709.699774:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+9:3303:x1876219371101952:12345-0@lo:6:osp-syn-0-0.0 Request processed in 6337us (6838us total) trans 4294967343 rc 0/0 00000100:00100000:0.0:1789302709.699783:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00010000:00010000:0.0:1789302709.699810:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302709.699817:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699827:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43ec00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302709.699832:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699842:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699857:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699864:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000025e62f6b/0x64fe43095184d3c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.699883:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302709.699885:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302709.699887:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789302709.699924:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f1e6ac380/2, rc 0, transno 4294967343 00000100:00100000:0.0:1789302709.699939:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371101952:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1789302709.700137:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302709.700148:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 13 pid: 6697 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.700160:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 13 pid: 6697 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.700176:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302709.700188:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x11:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302709.700193:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.700202:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.700217:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302709.700247:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350293760:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1612us (1855us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.700255:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 268 00000100:00080000:2.0:1789302709.700279:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789302709.700618:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004d843203/0x64fe43095184d1fa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x11:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01dd expref: 12 pid: 6697 timeout: 646 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302709.700632:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371102464:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302709.700869:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.700889:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.700897:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.700906:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.700912:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.700938:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302709.701241:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371102208:12345-0@lo:6:osp-syn-1-0.0 Request processed in 6677us (7953us total) trans 4294967345 rc 0/0 00000100:00100000:3.0:1789302709.701250:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00010000:00010000:0.0:1789302709.701309:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302709.701318:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.701326:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e02255800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302709.701332:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.701341:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.701357:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.701363:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c61bc9f9/0x64fe43095184d3d6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x11:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.701380:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302709.701382:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302709.701383:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302709.701946:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.702047:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350294016 00000100:00100000:1.0:1789302709.702056:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350294016 00000100:00100000:1.0:1789302709.702107:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 220 00000100:00100000:1.0:1789302709.702117:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2347:x1876219350294016:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000004:00080000:2.0:1789302709.702921:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ad44380/2, rc 0, transno 4294967345 00000100:00100000:2.0:1789302709.702931:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371102208:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789302709.776053:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371102592:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302709.776139:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302709.776212:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371102592 00000100:00100000:1.0:1789302709.776219:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371102592 00000100:00100000:1.0:1789302709.776270:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:1.0:1789302709.776278:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371102592:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302709.776301:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 78184448 pending 8519680 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302709.776345:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371102592:12345-0@lo:13:osp-pre-1-0.0 Request processed in 67us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.776352:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:1.0:1789302709.776391:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371102592:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789302709.793050:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967340 is committed 00000001:00080000:3.0:1789302709.793073:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967343 is committed 00000001:00080000:3.0:1789302709.793104:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967344 is committed 00000001:00080000:3.0:1789302709.793116:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967345 is committed 00000100:00100000:1.0:1789302709.793749:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350294016:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 91627us (91800us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.793761:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 220 00000100:00100000:1.0:1789302709.807367:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.807456:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350294272 00000100:00100000:0.0:1789302709.807467:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350294272 00000100:00100000:0.0:1789302709.807555:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 869 00000100:00100000:0.0:1789302709.807565:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350294272:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.807614:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302709.807640:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255400/0x64fe43095184d3b3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0262 expref: 38 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.807673:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:0.0:1789302709.807688:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302709.807720:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.807734:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.807744:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302709.807912:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000950d1825 reply state 00000000949fcdfb transno 4294967872 ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.807922:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302709.808111:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:24618:x1876219350294272:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 546us (750us total) trans 4294967872 rc 0/0 00000100:00100000:0.0:1789302709.808118:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 869 00010000:00010000:1.0:1789302709.809537:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.809552:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.809559:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.809566:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43ec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302709.809571:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.809578:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.809604:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:1.0:1789302709.809607:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302709.809617:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 2/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.809624:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43ec00/0x64fe43095184d3dd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.810363:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.810546:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350294400 00000100:00100000:2.0:1789302709.810556:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350294400 00000100:00100000:2.0:1789302709.810610:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 870 00000100:00100000:2.0:1789302709.810621:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350294400:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302709.810642:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302709.810658:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0277 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.810720:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.810735:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.810781:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:2.0:1789302709.810784:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302709.810786:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e02254200, rc 0) 00000100:00100000:2.0:1789302709.810840:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:24618:x1876219350294400:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 221us (481us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.810847:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 870 00000100:00100000:2.0:1789302709.815392:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.815486:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350294528 00000100:00100000:0.0:1789302709.815496:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350294528 00000100:00100000:0.0:1789302709.815579:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 221 00000100:00100000:0.0:1789302709.815590:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350294528:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1789302709.815864:0:6695:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x12:0x0] updating lvb size from disk: 0 -> 5242880 00002000:00010000:0.0:1789302709.815868:0:6695:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x12:0x0] updating lvb mtime from disk: 0 -> 1789302707 00002000:00010000:0.0:1789302709.815871:0:6695:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x12:0x0] updating lvb atime from disk: 0 -> 1789302707 00002000:00010000:0.0:1789302709.815873:0:6695:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x12:0x0] updating lvb ctime from disk: 0 -> 1789302707 00002000:00010000:0.0:1789302709.815875:0:6695:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x12:0x0] updating lvb blocks from disk: 1 -> 10247 00000100:00100000:0.0:1789302709.815952:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350294528:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 364us (569us total) trans 4294967346 rc 0/0 00000100:00100000:0.0:1789302709.815960:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 221 00000100:00100000:3.0:1789302709.820152:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.820307:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350294784 00000100:00100000:0.0:1789302709.820317:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350294784 00000100:00100000:0.0:1789302709.820374:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 871 00000100:00100000:0.0:1789302709.820384:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:24618:x1876219350294784:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.820445:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.820462:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 39 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.820471:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 4/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.820613:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 39 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.820627:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 39 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.820641:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371102720:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302709.820779:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599c00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.823175:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.823257:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350294912 00000100:00100000:1.0:1789302709.823270:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350294912 00010000:00010000:1.0:1789302709.823321:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 40 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.823405:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 270 00000100:00100000:1.0:1789302709.823416:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:2347:x1876219350294912:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302709.823435:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302709.823444:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 40 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.823459:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 40 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.823486:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:1.0:1789302709.823492:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.823545:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.823567:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.823592:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302709.823659:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:2347:x1876219350294912:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 245us (500us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.823666:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 270 00010000:00010000:0.0:1789302709.823812:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d382 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x16:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a024d expref: 39 pid: 5845 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.823844:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371102720:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302709.823886:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823896:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823903:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823933:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823940:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823947:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823953:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43f000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302709.823957:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823963:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823979:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:0.0:1789302709.823981:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1789302709.823986:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 2/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.823992:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3eb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.824146:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350294784:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 3761us (4001us total) trans 0 rc -22/-22 00000100:00100000:0.0:1789302709.824157:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 871 00000100:00100000:3.0:1789302709.829206:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.829328:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350295040 00000100:00100000:2.0:1789302709.829424:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350295040 00000100:00100000:2.0:1789302709.829606:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 222 00000100:00100000:3.0:1789302709.829703:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350295040:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302709.829770:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.829814:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.829825:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302709.829831:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302709.829836:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.829883:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.829900:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 11 pid: 8571 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302709.829920:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c92a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371102848:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302709.830036:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f050b2840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302709.834184:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c92a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371102848:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302709.834217:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302709.846181:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.847463:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350295168 00000100:00100000:1.0:1789302709.847630:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350295168 00010000:00010000:1.0:1789302709.847701:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 12 pid: 8571 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.847869:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 271 00000100:00100000:1.0:1789302709.847882:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350295168:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302709.847903:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302709.847935:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 12 pid: 8571 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.847953:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 12 pid: 8571 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.847981:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b11e5413/0x64fe43095184d3c1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0270 expref: 12 pid: 8571 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.848002:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302709.848007:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302709.848034:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.848047:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.848087:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302709.848157:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.848180:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.848183:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350295168:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 301us (2013us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.848208:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 271 00010000:00010000:0.0:1789302709.848240:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.848253:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.848272:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.848280:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302709.848286:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.848315:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.848336:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.848345:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184d3f2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.848469:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350295040:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 18848us (19273us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302709.848479:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 222 00000100:00100000:3.0:1789302709.850587:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.853380:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350295424 00000100:00100000:0.0:1789302709.853396:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350295424 00000100:00100000:0.0:1789302709.853460:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 872 00000100:00100000:0.0:1789302709.853469:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:24618:x1876219350295424:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302709.853561:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.853580:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.853593:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.853808:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.853832:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.853841:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.853873:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.853887:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 39 pid: 6919 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.853902:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371102976:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302709.853990:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599c00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.856641:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.856718:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350295552 00000100:00100000:1.0:1789302709.856735:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350295552 00010000:00010000:1.0:1789302709.856808:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 40 pid: 6919 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302709.856811:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371102976:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302709.856851:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.856879:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 272 00000100:00100000:1.0:1789302709.856893:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:2348:x1876219350295552:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302709.856913:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302709.856922:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 40 pid: 6919 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.856952:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 40 pid: 6919 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.856980:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02254200/0x64fe43095184d3e4 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0277 expref: 40 pid: 6919 timeout: 646 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.857001:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:1.0:1789302709.857023:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.857034:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.857043:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.857076:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302709.857103:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.857115:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.857153:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b2c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2348:x1876219350295552:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 261us (518us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302709.857163:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 272 00000001:00080000:0.0:1789302709.857818:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967873 00000001:00080000:0.0:1789302709.857981:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967874 00000001:00080000:0.0:1789302709.858028:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967875 00000100:00100000:0.0:1789302709.858112:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371103104:0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789302709.858181:0:5845:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302709.858273:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2900 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.864627:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371103104 00000100:00100000:3.0:1789302709.864644:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371103104 00000100:00100000:3.0:1789302709.864766:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 31 00000100:00100000:3.0:1789302709.864778:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371103104:12345-0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789302709.864913:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371103104:12345-0@lo:1000:mdt00_002.0 Request processed in 137us (6730us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302709.864936:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 31 00000100:00100000:1.0:1789302709.865013:0:5845:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967345 gen 2 00000004:00080000:1.0:1789302709.865033:0:5845:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ad44380, transno 4294967345 00000100:00100000:1.0:1789302709.865055:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371103104:0@lo:1000:mdt00_002.0 00000001:00080000:1.0:1789302709.865622:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967876 00000100:00100000:1.0:1789302709.865710:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371103232:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789302709.865756:0:5845:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302709.871010:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599c00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302709.875002:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371103232 00000100:00100000:0.0:1789302709.875018:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371103232 00000100:00100000:0.0:1789302709.875081:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 32 00000100:00100000:0.0:1789302709.875090:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371103232:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789302709.875404:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5845:x1876219371103232:12345-0@lo:1000:mdt00_002.0 Request processed in 314us (9646us total) trans 4294967347 rc 0/0 00000100:00100000:0.0:1789302709.875413:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 32 00000100:00100000:1.0:1789302709.875613:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb8e00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371103232:0@lo:1000:mdt00_002.0 00000001:00080000:1.0:1789302709.875722:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967877 00010000:00010000:1.0:1789302709.875761:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875772:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875781:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875800:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302709.875804:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875813:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875829:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501700) 00010000:00010000:1.0:1789302709.875833:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302709.875838:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875845:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43e000/0x64fe43095184d400 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x16:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875877:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875883:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875889:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875894:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302709.875896:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875902:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875920:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:1.0:1789302709.875923:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302709.875930:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 82 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875941:0:5845:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 82 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875951:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 82 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.875983:0:5845:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 82 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.875997:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371103360:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302709.876068:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599c00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302709.880613:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.880704:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350295680 00000100:00100000:2.0:1789302709.880714:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350295680 00000100:00100000:2.0:1789302709.880773:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 178 00000100:00100000:2.0:1789302709.880783:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ed050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:24617:x1876219350295680:12345-192.168.203.11@tcp:49:md5sum.0 00000100:00100000:2.0:1789302709.880907:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ed050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:24617:x1876219350295680:12345-192.168.203.11@tcp:49:md5sum.0 Request processed in 125us (302us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.880915:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 178 00000100:00100000:1.0:1789302709.882718:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbaa00 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371103360:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302709.882747:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882824:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184d2f6 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882861:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882875:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882901:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882913:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882919:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882926:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882932:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302709.882936:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882943:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882955:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:1.0:1789302709.882959:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302709.882963:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882969:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d407 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882980:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882986:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.882998:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.883003:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43f000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302709.883006:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.883013:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.883021:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:1.0:1789302709.883024:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302709.883027:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302709.883033:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f000/0x64fe43095184d3f9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302709.883477:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:24618:x1876219350295424:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 30006us (32894us total) trans 4294967878 rc 0/0 00000100:00100000:1.0:1789302709.883487:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 872 00000020:00080000:1.0:1789302709.883493:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:0.0:1789302709.887552:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302709.887562:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302709.888097:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1789302709.888107:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302709.891032:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.891992:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350295808 00000100:00100000:1.0:1789302709.892003:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350295808 00000100:00100000:2.0:1789302709.892122:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 223 00000100:00100000:2.0:1789302709.892133:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:24617:x1876219350295808:12345-192.168.203.11@tcp:101:md5sum.0 00010000:00010000:2.0:1789302709.892152:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302709.892175:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a027e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302709.892202:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x12:0x0] initial LVB size: 5242880, mtime: 0x6aa697b3, atime: 0x6aa697b3, ctime: 0x6aa697b3, blocks: 0x2807 00010000:00010000:2.0:1789302709.892237:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=546s v=(0 0) 00010000:00010000:2.0:1789302709.892242:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302709.892253:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.892282:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.892296:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e02255e00, rc 0) 00000100:00100000:3.0:1789302709.892854:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302709.893090:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350296064 00000100:00100000:0.0:1789302709.893097:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350296064 00000100:00100000:0.0:1789302709.893131:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 179 00000100:00100000:0.0:1789302709.893139:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:24618:x1876219350296064:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1789302709.894223:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x16:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1789302709.896271:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967879 00010000:00010000:0.0:1789302709.896459:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.896476:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.896488:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.896497:0:5846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.896607:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.896613:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302709.896688:0:5846:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302709.896791:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:24618:x1876219350296064:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 3649us (3936us total) trans 4294967880 rc 0/0 00000100:00100000:0.0:1789302709.896800:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 179 00000020:00080000:0.0:1789302709.896806:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789302709.897890:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:24617:x1876219350295808:12345-192.168.203.11@tcp:101:md5sum.0 Request processed in 5753us (6863us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302709.897901:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 223 00010000:00010000:3.0:1789302709.899308:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302709.899322:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.899342:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f1d43f200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302709.899348:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.899359:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.899378:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x16:0x0].0x0 (ffff960f03501200) 00010000:00010000:3.0:1789302709.899381:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302709.899386:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302709.899393:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f1d43f200/0x64fe43095184d415 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x16:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302709.899848:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302709.899852:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302709.899854:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302709.902663:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.902764:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350296192 00000100:00100000:2.0:1789302709.902775:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350296192 00002000:00100000:2.0:1789302709.902826:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x12:0x0] (0->32767) req@ffff960f41c90380 x1876219350296192/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302709.902838:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c90380 with x1876219350296192 ext(0->32767) 00002000:00010000:2.0:1789302709.902844:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f41c90380 00000100:00100000:2.0:1789302709.902877:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 82 00000100:00100000:2.0:1789302709.902888:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350296192:12345-192.168.203.11@tcp:3:md5sum.0 00000100:00100000:2.0:1789302709.908238:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350296192:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 5343us (5575us total) trans 0 rc 32768/32768 00000100:00100000:2.0:1789302709.908251:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 82 00002000:00100000:2.0:1789302709.908283:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x12:0x0] (0->32767) req@ffff960f41c90380 x1876219350296192/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/440 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 32768/32768 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302709.908294:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c90380 with x1876219350296192 ext(0->32767) 00002000:00010000:2.0:1789302709.908300:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f41c90380 00010000:00010000:0.0:1789302709.908401:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302709.914378:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.914571:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350296320 00000100:00100000:1.0:1789302709.914586:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350296320 00002000:00100000:1.0:1789302709.914655:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x12:0x0] (32768->1048575) req@ffff960f41c91c00 x1876219350296320/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.914668:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c91c00 with x1876219350296320 ext(32768->1048575) 00002000:00010000:1.0:1789302709.914735:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f41c91c00 00000100:00100000:1.0:1789302709.914833:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 83 00000100:00100000:1.0:1789302709.914844:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350296320:12345-192.168.203.11@tcp:3:md5sum.0 00000100:00100000:1.0:1789302709.932593:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350296320:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 17646us (18126us total) trans 0 rc 1015808/1015808 00000100:00100000:1.0:1789302709.932611:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 83 00002000:00100000:1.0:1789302709.932628:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x12:0x0] (32768->1048575) req@ffff960f41c91c00 x1876219350296320/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/440 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 1015808/1015808 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.932652:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c91c00 with x1876219350296320 ext(32768->1048575) 00002000:00010000:1.0:1789302709.932660:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f41c91c00 00000100:00100000:2.0:1789302709.958271:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.958314:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302709.958383:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350296448 00000100:00100000:1.0:1789302709.958402:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350296448 00000100:00100000:3.0:1789302709.958463:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350296576 00000100:00100000:3.0:1789302709.958469:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350296576 00002000:00100000:3.0:1789302709.958496:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x12:0x0] (2097152->3145727) req@ffff960e03981180 x1876219350296576/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302709.958549:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03981180 with x1876219350296576 ext(2097152->3145727) 00002000:00010000:3.0:1789302709.958574:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03981180 00000100:00100000:3.0:1789302709.958610:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 84 00000100:00100000:3.0:1789302709.958621:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350296576:12345-192.168.203.11@tcp:3:md5sum.0 00002000:00100000:1.0:1789302709.958856:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x12:0x0] (1048576->2097151) req@ffff960e03981f80 x1876219350296448/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.958868:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03981f80 with x1876219350296448 ext(1048576->2097151) 00002000:00010000:1.0:1789302709.958872:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03981f80 00000100:00100000:1.0:1789302709.958903:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 85 00000100:00100000:1.0:1789302709.958910:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350296448:12345-192.168.203.11@tcp:3:md5sum.0 00000100:00100000:2.0:1789302709.974470:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302709.974554:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302709.974664:0:6700:0:(service.c:3467:ptlrpc_start_thread()) ost_io[0] started 4 min 3 max 116 00000100:00100000:3.0:1789302709.974691:0:6700:0:(service.c:3526:ptlrpc_start_thread()) starting thread 'll_ost_io00_004' 00000100:00100000:1.0:1789302709.974863:0:6702:0:(service.c:3467:ptlrpc_start_thread()) ost_io[0] started 4 min 3 max 116 00000100:00100000:1.0:1789302709.974877:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350296832 00000100:00100000:1.0:1789302709.974898:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350296832 00002000:00100000:1.0:1789302709.974965:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x12:0x0] (4194304->5242879) req@ffff960e03981500 x1876219350296832/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.974978:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03981500 with x1876219350296832 ext(4194304->5242879) 00002000:00010000:1.0:1789302709.975014:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03981500 00000100:00100000:1.0:1789302709.975035:0:6702:0:(service.c:3467:ptlrpc_start_thread()) ost_io[0] started 4 min 3 max 116 00000100:00100000:1.0:1789302709.975040:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350296704 00000100:00100000:1.0:1789302709.975043:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350296704 00002000:00100000:1.0:1789302709.975058:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x12:0x0] (3145728->4194303) req@ffff960e03983480 x1876219350296704/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/0 e 0 to 0 dl 1789302720 ref 1 fl New:/600/ffffffff rc 0/-1 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.975066:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03983480 with x1876219350296704 ext(3145728->4194303) 00002000:00010000:1.0:1789302709.975069:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03983480 00000100:00100000:3.0:1789302709.976605:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 86 00000100:00100000:3.0:1789302709.976625:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350296832:12345-192.168.203.11@tcp:3:md5sum.0 00000100:00100000:1.0:1789302709.999553:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350296576:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 40868us (41172us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789302709.999572:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 84 00002000:00100000:1.0:1789302709.999590:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x12:0x0] (2097152->3145727) req@ffff960e03981180 x1876219350296576/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/440 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302709.999603:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03981180 with x1876219350296576 ext(2097152->3145727) 00002000:00010000:1.0:1789302709.999611:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03981180 00000100:00100000:1.0:1789302709.999652:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 87 00000100:00100000:1.0:1789302709.999667:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350296704:12345-192.168.203.11@tcp:3:md5sum.0 00000100:00100000:1.0:1789302710.015206:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350296448:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 56272us (56934us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789302710.015224:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 85 00002000:00100000:1.0:1789302710.015242:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x12:0x0] (1048576->2097151) req@ffff960e03981f80 x1876219350296448/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/440 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302710.015255:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03981f80 with x1876219350296448 ext(1048576->2097151) 00002000:00010000:1.0:1789302710.015264:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03981f80 00000100:00100000:0.0:1789302710.041821:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2347:x1876219350296832:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 65192us (67349us total) trans 0 rc 1048576/1048576 00000100:00100000:0.0:1789302710.041834:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 86 00002000:00100000:0.0:1789302710.041846:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x12:0x0] (4194304->5242879) req@ffff960e03981500 x1876219350296832/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/440 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302710.041855:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03981500 with x1876219350296832 ext(4194304->5242879) 00002000:00010000:0.0:1789302710.041861:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03981500 00000100:00100000:1.0:1789302710.048290:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350296704:12345-192.168.203.11@tcp:3:md5sum.0 Request processed in 48618us (73725us total) trans 0 rc 1048576/1048576 00000100:00100000:1.0:1789302710.048304:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 87 00002000:00100000:1.0:1789302710.048318:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x12:0x0] (3145728->4194303) req@ffff960e03983480 x1876219350296704/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:285/0 lens 488/440 e 0 to 0 dl 1789302720 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'md5sum.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302710.048327:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03983480 with x1876219350296704 ext(3145728->4194303) 00002000:00010000:1.0:1789302710.048343:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03983480 00000100:00100000:1.0:1789302710.083370:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302710.083590:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350297088 00000100:00100000:0.0:1789302710.083600:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350297088 00000100:00100000:0.0:1789302710.083666:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 180 00000100:00100000:0.0:1789302710.083676:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:24617:x1876219350297088:12345-192.168.203.11@tcp:35:md5sum.0 00000100:00100000:0.0:1789302710.084153:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:24617:x1876219350297088:12345-192.168.203.11@tcp:35:md5sum.0 Request processed in 477us (790us total) trans 4294967881 rc 0/0 00000100:00100000:0.0:1789302710.084162:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 180 00000100:00100000:0.0:1789302710.286229:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371103488:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302710.286292:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302710.286473:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371103488 00000100:00100000:2.0:1789302710.286480:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371103488 00000100:00100000:2.0:1789302710.286544:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 149 00000100:00100000:2.0:1789302710.286552:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3303:x1876219371103488:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302710.286582:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 79888384 pending 8519680 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302710.286624:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3303:x1876219371103488:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302710.286630:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 149 00000100:00100000:3.0:1789302710.286716:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371103488:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302710.927612:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302711.949659:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302711.950210:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967343 is committed 00000001:00080000:1.0:1789302711.950228:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967340 is committed 00010000:00010000:0.0:1789302712.972873:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302714.007927:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302714.250919:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302714.251615:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350297216 00000100:00100000:1.0:1789302714.251630:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350297216 00000100:00100000:1.0:1789302714.251715:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 873 00000100:00100000:1.0:1789302714.251739:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25036:x1876219350297216:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302714.251771:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302714.251796:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d41c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0285 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.251891:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e400/0x64fe43095184d423 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.251909:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e400/0x64fe43095184d423 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.251923:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e400/0x64fe43095184d423 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302714.251966:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00000004:00200000:1.0:1789302714.251977:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302714.252003:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e400/0x64fe43095184d423 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.252034:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d41c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0285 expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.252052:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e400/0x64fe43095184d423 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0285 expref: 85 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.252062:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302714.252066:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302714.252070:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7e400, rc 0) 00000100:00100000:1.0:1789302714.252145:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25036:x1876219350297216:12345-192.168.203.11@tcp:101:rm.0 Request processed in 419us (1236us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.252152:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 873 00000100:00080000:2.0:1789302714.252574:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302714.258273:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302714.260487:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350297472 00000100:00100000:3.0:1789302714.260525:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350297472 00000100:00100000:3.0:1789302714.260706:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 874 00000100:00100000:3.0:1789302714.260777:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25036:x1876219350297472:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302714.261022:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302714.261051:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d42a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a028c expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302714.261171:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:3.0:1789302714.261327:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d42a lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a028c expref: 87 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302714.261344:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:3.0:1789302714.261347:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302714.261354:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d42a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a028c expref: 87 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302714.261368:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:3.0:1789302714.261527:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25036:x1876219350297472:12345-192.168.203.11@tcp:101:rm.0 Request processed in 810us (3268us total) trans 4294967882 rc 301/301 00000100:00100000:3.0:1789302714.263322:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 874 00000100:00100000:3.0:1789302714.266180:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302714.266489:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350297600 00000100:00100000:0.0:1789302714.266538:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350297600 00000100:00100000:0.0:1789302714.266602:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 181 00000100:00100000:0.0:1789302714.266907:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25036:x1876219350297600:12345-192.168.203.11@tcp:37:rm.0 00000100:00100000:2.0:1789302714.272222:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25036:x1876219350297600:12345-192.168.203.11@tcp:37:rm.0 Request processed in 5305us (6039us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302714.272239:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 181 00000100:00100000:2.0:1789302714.277423:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302714.277551:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350297856 00000100:00100000:0.0:1789302714.277564:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350297856 00000100:00100000:0.0:1789302714.277754:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:0.0:1789302714.277765:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25036:x1876219350297856:12345-192.168.203.11@tcp:35:rm.0 00000100:00100000:0.0:1789302714.278103:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:25036:x1876219350297856:12345-192.168.203.11@tcp:35:rm.0 Request processed in 339us (692us total) trans 4294967883 rc 0/0 00000100:00100000:0.0:1789302714.278111:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 182 00000100:00100000:3.0:1789302714.280277:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302714.280443:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350298112 00000100:00100000:1.0:1789302714.280455:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350298112 00000100:00100000:1.0:1789302714.280608:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 875 00000100:00100000:1.0:1789302714.280619:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:25036:x1876219350298112:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302714.280657:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302714.280781:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d438 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0293 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.280908:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d43f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.280925:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d43f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.280952:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d43f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302714.280968:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:1.0:1789302714.281112:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d438 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0293 expref: 90 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.281130:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d43f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0293 expref: 89 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.281140:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302714.281142:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:1.0:1789302714.281147:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f800, rc 0) 00000100:00100000:1.0:1789302714.281381:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25036:x1876219350298112:12345-192.168.203.11@tcp:101:rm.0 Request processed in 763us (1109us total) trans 4294967884 rc 0/0 00000100:00100000:1.0:1789302714.281390:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 875 00000100:00100000:1.0:1789302714.284717:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302714.284848:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350298240 00000100:00100000:2.0:1789302714.284869:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350298240 00000100:00100000:2.0:1789302714.284930:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 876 00000100:00100000:2.0:1789302714.284942:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25036:x1876219350298240:12345-192.168.203.11@tcp:41:rm.0 00000004:00000024:2.0:1789302714.285017:0:5844:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 7715422208 avail 7688159232 00000100:00100000:2.0:1789302714.285112:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25036:x1876219350298240:12345-192.168.203.11@tcp:41:rm.0 Request processed in 171us (408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302714.285120:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 876 00000100:00100000:2.0:1789302714.287347:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302714.287481:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350298368 00000100:00100000:0.0:1789302714.287494:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350298368 00000100:00100000:0.0:1789302714.287578:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 877 00000100:00100000:0.0:1789302714.287589:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25036:x1876219350298368:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302714.287614:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302714.287643:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d44d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a029a expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.287742:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.287761:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.287788:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.287806:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.287814:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.287821:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288243:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d462 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288252:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288259:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302714.288300:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x38:0x0]: returning size 5242880 00000004:00200000:0.0:1789302714.288305:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302714.288326:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288339:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d462 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288347:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288364:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288371:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288378:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302714.288383:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288391:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288407:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302714.288410:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302714.288416:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288422:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d454 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288434:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288440:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288445:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288450:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302714.288459:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288465:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288473:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff960f03501b00) 00010000:00010000:0.0:1789302714.288476:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302714.288479:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288484:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d45b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288545:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d44d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a029a expref: 91 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288561:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d462 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a029a expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.288569:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:0.0:1789302714.288571:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302714.288573:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302714.288662:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:25036:x1876219350298368:12345-192.168.203.11@tcp:101:rm.0 Request processed in 1074us (1327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302714.288670:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 877 00000100:00100000:3.0:1789302714.291672:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302714.297858:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350298624 00000100:00100000:0.0:1789302714.297880:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350298624 00000100:00100000:0.0:1789302714.297965:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 878 00000100:00100000:0.0:1789302714.297979:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:25036:x1876219350298624:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:0.0:1789302714.298066:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789302714.298102:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d462 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x38:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a029a expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298125:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e02255c00/0x64fe43095184d2fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x38:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a022a expref: 89 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298138:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f0cdf2c00) 00010000:00010000:0.0:1789302714.298158:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e400/0x64fe43095184d423 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0285 expref: 88 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298170:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302714.298175:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302714.298201:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298217:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298228:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298245:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298254:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298260:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298668:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298678:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.298685:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789302714.300912:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].9, 64 off8840 00000004:00080000:0.0:1789302714.300928:0:5845:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 551311425519ns, next at 552311421756ns, rc = 0 00010000:00010000:0.0:1789302714.301015:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301033:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301046:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301055:0:5845:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301061:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 4/0,1 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301067:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301087:0:5845:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789302714.301114:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301127:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301134:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301140:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302714.301146:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301156:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301172:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302714.301175:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302714.301181:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301188:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d477 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x38:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301209:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301215:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301222:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301227:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302714.301236:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301243:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301252:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302714.301255:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302714.301259:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301266:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d469 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x36:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302714.301276:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009d414268 reply state 00000000712725a7 transno 4294967885 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302714.301287:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302714.301473:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25036:x1876219350298624:12345-192.168.203.11@tcp:36:rm.0 Request processed in 3491us (9818us total) trans 4294967885 rc 0/0 00000100:00100000:0.0:1789302714.301485:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 878 00010000:00010000:0.0:1789302714.301567:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302714.301574:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301581:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302714.301584:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301591:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301612:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x38:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302714.301614:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302714.301618:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 1/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301623:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d47e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x38:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.301638:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302714.301641:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302714.301642:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302714.308091:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.308115:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.308125:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.308132:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302714.308138:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.308147:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.309132:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x361913b3 (ffff960f03501a00) 00010000:00010000:0.0:1789302714.309139:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302714.309148:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.309164:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d470 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x36:0x0].0x361913b3 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302714.319597:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302714.319680:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350298880 00000100:00100000:2.0:1789302714.319694:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350298880 00000100:00100000:2.0:1789302714.319782:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 183 00000100:00100000:2.0:1789302714.319795:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:25036:x1876219350298880:12345-192.168.203.11@tcp:35:rm.0 00000100:00100000:2.0:1789302714.320126:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25036:x1876219350298880:12345-192.168.203.11@tcp:35:rm.0 Request processed in 333us (544us total) trans 4294967886 rc 0/0 00000100:00100000:2.0:1789302714.320135:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 183 00000100:00100000:3.0:1789302714.328059:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302714.328211:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350299136 00000100:00100000:0.0:1789302714.328225:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350299136 00000100:00100000:0.0:1789302714.328298:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 879 00000100:00100000:0.0:1789302714.328311:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25036:x1876219350299136:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:0.0:1789302714.328385:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:0.0:1789302714.328420:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d43f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x36:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0293 expref: 89 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.328445:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d183 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01ac expref: 88 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.328458:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302714.328470:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e0221fc00/0x64fe43095184d167 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a01a5 expref: 87 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.328479:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302714.328483:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302714.328540:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.328558:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 44 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.328569:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.328631:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 44 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.328659:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 44 pid: 18427 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302714.328676:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371103616:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302714.328797:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0958ff00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302714.331130:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371103616:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302714.331167:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302714.331303:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302714.331358:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350299264 00000100:00100000:1.0:1789302714.331367:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350299264 00010000:00010000:1.0:1789302714.331429:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 45 pid: 18427 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302714.331477:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 273 00000100:00100000:1.0:1789302714.331489:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2347:x1876219350299264:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302714.331719:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302714.331729:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 45 pid: 18427 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.331756:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 45 pid: 18427 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.331784:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184d335 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0238 expref: 45 pid: 18427 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.331801:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:1.0:1789302714.331807:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.331817:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.331826:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.331853:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302714.331922:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350299264:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 435us (620us total) trans 0 rc 0/0 00000100:00080000:2.0:1789302714.331984:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302714.332048:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.332056:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.332085:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.332095:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.332104:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302714.333474:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 273 00010000:00010000:0.0:1789302714.333740:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.333768:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.333776:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.333822:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.333856:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 43 pid: 5845 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302714.333874:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371103744:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302714.334177:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0958ff00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302714.342169:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302714.342270:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371103744:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302714.342306:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302714.342572:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350299392 00000100:00100000:1.0:1789302714.342582:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350299392 00010000:00010000:1.0:1789302714.342627:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 44 pid: 5845 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302714.342673:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 274 00000100:00100000:1.0:1789302714.342683:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2348:x1876219350299392:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302714.342830:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302714.342840:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 44 pid: 5845 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.342854:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 44 pid: 5845 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.342881:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d600/0x64fe43095184d351 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x36:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a023f expref: 44 pid: 5845 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.342977:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302714.342982:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.342991:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.342998:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.343022:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302714.343043:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.343051:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302714.343096:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2348:x1876219350299392:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 413us (942us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.343103:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 274 00000004:00200000:0.0:1789302714.344979:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x36:0x0]: returning size 16384 00010000:00010000:0.0:1789302714.345006:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345027:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345036:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345042:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302714.345048:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345056:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345071:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x36:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302714.345074:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302714.345079:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345086:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d493 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345114:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345121:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345127:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345132:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302714.345140:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345147:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345155:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302714.345157:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302714.345160:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.345167:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d485 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302714.345182:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a788fd5b reply state 0000000002168f1a transno 4294967887 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302714.345192:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302714.345487:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:25036:x1876219350299136:12345-192.168.203.11@tcp:36:rm.0 Request processed in 17174us (17435us total) trans 4294967887 rc 0/0 00000100:00100000:0.0:1789302714.345497:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 879 00010000:00010000:1.0:1789302714.347580:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.347598:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.347627:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.347633:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302714.347639:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.347657:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.347837:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c6c0408 (ffff960f03501a00) 00010000:00010000:1.0:1789302714.347840:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302714.347846:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.347854:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d48c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c6c0408 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302714.381142:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302714.381208:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302714.381238:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371104000:0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.381349:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302714.384851:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371104000 00000100:00100000:3.0:1789302714.384876:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371104000 00000100:00100000:3.0:1789302714.384988:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:3.0:1789302714.385001:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371104000:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302714.385120:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371104000:12345-0@lo:400:kworker.0 Request processed in 122us (3771us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302714.385128:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:1.0:1789302714.385314:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371104000:0@lo:400:kworker.0 00000100:00100000:2.0:1789302714.386833:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371103872:0@lo:400:kworker.0 00000100:00100000:2.0:1789302714.386920:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302714.387087:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302714.387347:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371103872 00000100:00100000:1.0:1789302714.387358:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371103872 00000100:00100000:1.0:1789302714.387571:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 540 00000100:00100000:1.0:1789302714.387584:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371103872:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302714.387604:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350299520 00000100:00100000:3.0:1789302714.387610:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350299520 00000100:00100000:3.0:1789302714.387672:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 880 00000100:00100000:3.0:1789302714.387682:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:24590:x1876219350299520:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1789302714.387684:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371103872:12345-0@lo:400:kworker.0 Request processed in 103us (762us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.387712:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 540 00000100:00100000:1.0:1789302714.387897:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371103872:0@lo:400:kworker.0 00000100:00100000:3.0:1789302714.387951:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:24590:x1876219350299520:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 270us (865us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302714.387959:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 880 00000100:00100000:2.0:1789302714.388646:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371104384:0@lo:400:kworker.0 00000100:00100000:2.0:1789302714.388707:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302714.388943:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371104256:0@lo:400:kworker.0 00000100:00100000:3.0:1789302714.388982:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302714.389019:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302714.389025:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371104128:0@lo:400:kworker.0 00000100:00100000:3.0:1789302714.389048:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302714.389079:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371104384 00000100:00100000:3.0:1789302714.389086:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371104384 00000100:00100000:3.0:1789302714.389132:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371104128 00000100:00100000:3.0:1789302714.389136:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371104128 00000100:00100000:3.0:1789302714.389159:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 351 00000100:00100000:3.0:1789302714.389166:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371104384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.389342:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371104256 00000100:00100000:1.0:1789302714.389347:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371104256 00000100:00100000:1.0:1789302714.389380:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1789302714.389386:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371104256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.389426:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371104256:12345-0@lo:400:kworker.0 Request processed in 39us (443us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.389432:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1789302714.389470:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967343 gen 2 00000004:00080000:1.0:1789302714.389484:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967341 committed transno 4294967343 00000004:00080000:1.0:1789302714.389522:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f1e6af100, transno 4294967342 00000004:00080000:1.0:1789302714.389531:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f1e6ac380, transno 4294967343 00000100:00100000:1.0:1789302714.389538:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371104256:0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.390018:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371104512:0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.390057:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302714.390099:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371104512 00000100:00100000:1.0:1789302714.390104:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371104512 00000100:00100000:1.0:1789302714.390134:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1789302714.390142:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371104512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.390179:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371104512:12345-0@lo:400:kworker.0 Request processed in 38us (120us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.390185:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 331 00000100:00100000:1.0:1789302714.390216:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371104512:0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.390237:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1789302714.390256:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371104128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302714.390295:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371104128:12345-0@lo:400:kworker.0 Request processed in 41us (1246us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.390302:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1789302714.390330:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371104128:0@lo:400:kworker.0 00000100:00100000:3.0:1789302714.394194:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371104384:12345-0@lo:400:kworker.0 Request processed in 5013us (5472us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302714.394213:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 351 00000100:00080000:3.0:1789302714.394273:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302714.394280:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302714.394285:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302714.397201:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371104384:0@lo:400:kworker.0 00000100:00100000:3.0:1789302714.636273:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302714.636327:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302714.636359:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302714.636383:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302714.636408:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302714.636440:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350299648 00000100:00100000:3.0:1789302714.636452:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350299648 00000100:00100000:3.0:1789302714.636545:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 541 00000100:00100000:3.0:1789302714.636558:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350299648:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302714.636640:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350299776 00000100:00100000:0.0:1789302714.636655:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350299776 00000100:00100000:0.0:1789302714.636706:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350300032 00000100:00100000:0.0:1789302714.636710:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350300032 00000100:00100000:0.0:1789302714.636725:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350299904 00000100:00100000:0.0:1789302714.636728:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350299904 00000100:00100000:0.0:1789302714.636742:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350300160 00000100:00100000:0.0:1789302714.636745:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350300160 00000100:00100000:0.0:1789302714.636760:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 332 00000100:00100000:0.0:1789302714.636768:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350299776:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:2.0:1789302714.636896:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302714.636901:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302714.636923:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 333 00000100:00100000:2.0:1789302714.636933:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350300032:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302714.637014:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 334 00000100:00100000:1.0:1789302714.637024:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350299904:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302714.637107:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350299648:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 497us (692us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302714.637117:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 541 00000100:00100000:3.0:1789302714.637174:0:13439:0:(service.c:3467:ptlrpc_start_thread()) ost[0] started 5 min 3 max 116 00000100:00100000:3.0:1789302714.637189:0:13439:0:(service.c:3526:ptlrpc_start_thread()) starting thread 'll_ost00_005' 00000100:00100000:0.0:1789302714.640183:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350299776:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 3393us (3904us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302714.640202:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 332 00000100:00100000:0.0:1789302714.640259:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:0.0:1789302714.640276:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350300160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302714.640345:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350300160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 76us (3934us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302714.640352:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:2.0:1789302714.642678:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350300032:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5734us (6336us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302714.642694:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 333 00000100:00100000:1.0:1789302714.642879:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350299904:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5855us (6493us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.642888:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 334 00000100:00100000:2.0:1789302714.893205:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302714.893237:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371104640:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302714.893337:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302714.893425:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371104640 00000100:00100000:2.0:1789302714.893435:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371104640 00000100:00100000:1.0:1789302714.893599:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1789302714.893614:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371104640:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302714.893670:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 69664768 pending 0 free 3851419648 avail 3849322496 00000100:00100000:1.0:1789302714.893730:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371104640:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (393us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.893738:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:2.0:1789302714.893896:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371104640:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789302714.960825:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967887 is committed 00000004:00080000:0.0:1789302714.960847:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 10 committed 00000001:00080000:0.0:1789302714.960874:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967880 is committed 00000020:00080000:0.0:1789302714.960897:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 13 batchid updated 00000020:00080000:0.0:1789302714.960933:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f098a2b40 refcount 2 committed 0 result 0 batchid 14 00000020:00080000:0.0:1789302714.960940:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f1d936800 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f33ff9cc0 00000020:00080000:0.0:1789302714.960943:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f1d936600 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960e03939400 00000020:00080000:0.0:1789302714.960964:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 14 committed batchid 13 00000020:00080000:0.0:1789302714.961193:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 13 00000100:00100000:2.0:1789302714.962086:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371104768:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789302714.962180:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302714.962227:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371104768 00000100:00100000:0.0:1789302714.962237:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371104768 00000100:00100000:0.0:1789302714.962304:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1789302714.962315:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371104768:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1789302714.962351:0:8571:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:18 count 1 00010000:00010000:0.0:1789302714.962456:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 3/0,1 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.962530:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.962604:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=5s v=(0 0) 00010000:00010000:0.0:1789302714.962611:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302714.962617:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 4/0,1 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.962666:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.962684:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 12 pid: 6696 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302714.962699:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371104896:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302714.962784:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f053b3900 going to sleep for 11 seconds 00000001:00080000:1.0:1789302714.964601:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967347 is committed 00000001:00080000:1.0:1789302714.964623:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967346 is committed 00000100:00100000:0.0:1789302714.968585:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371104896:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302714.968631:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 3/0,1 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302714.969831:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302714.970048:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350300288 00000100:00100000:1.0:1789302714.970060:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350300288 00010000:00010000:1.0:1789302714.970116:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 13 pid: 6696 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302714.970171:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 275 00000100:00100000:1.0:1789302714.970188:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2345:x1876219350300288:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302714.970225:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302714.970233:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 13 pid: 6696 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.970250:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 13 pid: 6696 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.970279:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005c2c30cd/0x64fe43095184d40e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x12:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->32767) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a027e expref: 13 pid: 6696 timeout: 651 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.970302:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302714.970308:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x12:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302714.970316:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.970328:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.970371:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302714.970461:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350300288:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 275us (638us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302714.970469:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 275 00010000:00010000:0.0:1789302714.970963:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.970975:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.971000:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.971008:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302714.971035:0:8571:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302714.972827:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302714.972838:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.972850:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302714.972863:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.972874:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.972895:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 1/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.972900:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000041982ca/0x64fe43095184d49a lrc: 0/0,0 mode: --/PW res: [0x280000400:0x12:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302714.972921:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302714.972922:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302714.972923:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302714.973374:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371104768:12345-0@lo:6:osp-syn-1-0.0 Request processed in 11055us (11187us total) trans 4294967348 rc 0/0 00000100:00100000:0.0:1789302714.973386:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:2.0:1789302714.976384:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967347 gen 2 00000004:00080000:2.0:1789302714.976400:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967347 committed transno 4294967347 00000100:00100000:2.0:1789302714.976440:0:3303:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff960f3f7c9c00 x1876219371104768/t4294967348(4294967348) o6->lustre-OST0001-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789302730 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-1-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:2.0:1789302714.976456:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f7c9c00/2, rc 0, transno 4294967348 00000100:00100000:2.0:1789302714.976462:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371104768:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1789302715.020916:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302715.406103:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371105024:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302715.406204:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302715.406313:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371105024 00000100:00100000:0.0:1789302715.406325:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371105024 00000100:00100000:0.0:1789302715.406390:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 151 00000100:00100000:0.0:1789302715.406399:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371105024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302715.406439:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 71368704 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302715.406494:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371105024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (296us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302715.406523:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 151 00000100:00100000:3.0:1789302715.406613:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371105024:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1789302716.044720:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302716.081584:0:18617:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12d: Restore of a non archived, non released file must work ========================================================== 08:31:54 (1789302714) 00000100:00100000:3.0:1789302716.410363:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.410799:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350300416 00000100:00100000:0.0:1789302716.410811:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350300416 00000100:00100000:0.0:1789302716.410892:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 881 00000100:00100000:0.0:1789302716.410904:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:25220:x1876219350300416:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302716.410925:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.410951:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4a1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02a1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.411032:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.411046:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.411057:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302716.411093:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302716.411096:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302716.411114:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.411149:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4a1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 36 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.411167:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 35 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.411175:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501b00) 00010000:00010000:0.0:1789302716.411178:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302716.411181:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebe00, rc 0) 00000100:00080000:2.0:1789302716.411377:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302716.411419:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:25220:x1876219350300416:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 517us (1063us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302716.411428:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 881 00000100:00100000:2.0:1789302716.422747:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302716.422858:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350300544 00000100:00100000:3.0:1789302716.422868:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350300544 00000100:00100000:3.0:1789302716.422988:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 882 00000100:00100000:3.0:1789302716.422999:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+80:25220:x1876219350300544:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302716.423020:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302716.423039:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d4af lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02a8 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.423095:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d4b6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.423108:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d4b6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.423118:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d4b6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302716.423150:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302716.423153:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302716.423169:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d4b6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.423190:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d4af lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a8 expref: 82 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.423207:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d4b6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a8 expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.423227:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501b00) 00010000:00010000:3.0:1789302716.423230:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302716.423234:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:3.0:1789302716.423292:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:25220:x1876219350300544:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 296us (553us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302716.423299:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 882 00000100:00080000:2.0:1789302716.423485:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302716.437238:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.438311:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350300672 00000100:00100000:0.0:1789302716.438323:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350300672 00000100:00100000:0.0:1789302716.438403:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 883 00000100:00100000:0.0:1789302716.438415:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:25221:x1876219350300672:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1789302716.438574:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:25221:x1876219350300672:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 160us (1344us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302716.438581:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 883 00000100:00100000:1.0:1789302716.469982:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302716.470091:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350300800 00000100:00100000:2.0:1789302716.470115:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350300800 00000100:00100000:2.0:1789302716.470171:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 884 00000100:00100000:2.0:1789302716.470181:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:25222:x1876219350300800:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302716.470201:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302716.470222:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d4bd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02af expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470283:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470298:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470309:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470323:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470330:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470336:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470576:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470585:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470592:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470603:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302716.470608:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470616:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470628:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501b00) 00010000:00010000:2.0:1789302716.470631:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302716.470652:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470659:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d4c4 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470677:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470683:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470689:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470694:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302716.470697:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470703:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470719:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff960f03927a00) 00010000:00010000:2.0:1789302716.470722:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302716.470725:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470731:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d4cb lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470759:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d4bd lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02af expref: 82 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470769:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501b00) 00010000:00010000:2.0:1789302716.470771:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302716.470775:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d4bd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02af expref: 82 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.470784:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfc00, rc 0) 00000100:00100000:2.0:1789302716.470842:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:25222:x1876219350300800:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 662us (868us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302716.470850:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 884 00000100:00100000:2.0:1789302716.489671:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.489838:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301056 00000100:00100000:0.0:1789302716.489853:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301056 00000100:00100000:0.0:1789302716.489929:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 885 00000100:00100000:0.0:1789302716.489939:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:25222:x1876219350301056:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:0.0:1789302716.489993:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302716.490017:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d4b6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a8 expref: 81 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.490039:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501b00) 00010000:00010000:0.0:1789302716.490043:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302716.490238:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.490253:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 34 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.490260:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.490291:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 34 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.490305:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 34 pid: 5845 timeout: 653 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302716.490329:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371105152:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302716.490653:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6b40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302716.498107:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371105152:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302716.498138:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302716.499850:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302716.500192:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301184 00000100:00100000:1.0:1789302716.500202:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301184 00010000:00010000:1.0:1789302716.500245:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 35 pid: 5845 timeout: 653 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302716.500286:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 276 00000100:00100000:1.0:1789302716.500297:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:2347:x1876219350301184:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302716.500312:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302716.500318:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 35 pid: 5845 timeout: 653 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.500349:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 35 pid: 5845 timeout: 653 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.500385:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d4a8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02a1 expref: 35 pid: 5845 timeout: 653 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.500402:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501b00) 00010000:00010000:1.0:1789302716.500407:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.500417:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.500426:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.500448:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302716.500524:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:2347:x1876219350301184:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 205us (656us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302716.500531:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 276 00010000:00010000:0.0:1789302716.501029:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.501038:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.501061:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.501071:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.501079:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302716.502321:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00010000:00010000:0.0:1789302716.502335:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.502344:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.502351:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.503682:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.503688:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.503698:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.503895:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501b00) 00010000:00010000:0.0:1789302716.503899:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.503904:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.503912:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4d2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302716.503933:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000029cd782 reply state 00000000593d08a5 transno 4294967888 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302716.503942:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302716.504132:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:25222:x1876219350301056:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 14189us (14473us total) trans 4294967888 rc 0/0 00000100:00100000:0.0:1789302716.504141:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 885 00010000:00010000:0.0:1789302716.514737:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.514755:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.514764:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.514789:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.514794:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.514802:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.514818:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff960f03501d00) 00010000:00010000:0.0:1789302716.514821:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.514826:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.514833:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4d9 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302716.523822:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.524575:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301312 00000100:00100000:0.0:1789302716.524586:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301312 00000100:00100000:0.0:1789302716.524655:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 886 00000100:00100000:0.0:1789302716.524677:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+81:25223:x1876219350301312:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302716.524701:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.524725:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4e0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02b6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.524792:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4e7 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.524805:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.524835:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302716.524871:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302716.524875:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302716.524894:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4e7 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.524920:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4e0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02b6 expref: 83 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.524938:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4e7 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02b6 expref: 82 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.524946:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302716.524949:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302716.524953:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb000, rc 0) 00000100:00100000:0.0:1789302716.525018:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:25223:x1876219350301312:12345-192.168.203.11@tcp:101:bash.0 Request processed in 353us (1205us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302716.525024:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 886 00000100:00100000:2.0:1789302716.536874:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.536967:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301440 00000100:00100000:0.0:1789302716.536976:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301440 00000100:00100000:0.0:1789302716.537030:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 887 00000100:00100000:0.0:1789302716.537039:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+82:25223:x1876219350301440:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302716.537059:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.537079:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4ee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02bd expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537160:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537174:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537186:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537200:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537207:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537214:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537302:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d503 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537310:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d503 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537317:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d503 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302716.537343:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000004:00200000:0.0:1789302716.537347:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302716.537362:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d503 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537455:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537461:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537468:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537474:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.537479:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537487:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537498:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302716.537536:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.537541:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537548:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d4f5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537561:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537568:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537574:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537585:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.537588:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537593:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537601:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff960f03501b00) 00010000:00010000:0.0:1789302716.537603:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.537605:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537611:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d4fc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537652:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d4ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02bd expref: 84 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537667:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb200/0x64fe43095184d503 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02bd expref: 83 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.537673:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302716.537675:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302716.537677:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb200, rc 0) 00000100:00100000:0.0:1789302716.538002:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25223:x1876219350301440:12345-192.168.203.11@tcp:101:bash.0 Request processed in 962us (1134us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302716.538009:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 887 00000100:00100000:1.0:1789302716.542997:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.543073:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301568 00000100:00100000:0.0:1789302716.543096:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301568 00000100:00100000:0.0:1789302716.543157:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 888 00000100:00100000:0.0:1789302716.543169:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25223:x1876219350301568:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302716.543192:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.543215:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d50a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02c4 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543286:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543300:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543312:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543327:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543334:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543340:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543416:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543424:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543431:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543445:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.543450:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543458:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543470:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302716.543474:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.543480:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543486:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d511 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543532:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543539:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543546:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543551:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebe00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.543555:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543561:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543581:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f03501600) 00010000:00010000:0.0:1789302716.543584:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.543588:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543594:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d518 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543629:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d50a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02c4 expref: 84 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543653:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302716.543655:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.543660:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d50a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02c4 expref: 84 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.543681:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:0.0:1789302716.543744:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25223:x1876219350301568:12345-192.168.203.11@tcp:101:bash.0 Request processed in 577us (758us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302716.543752:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 888 00000100:00100000:2.0:1789302716.570942:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302716.571046:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301696 00000100:00100000:1.0:1789302716.571056:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301696 00000100:00100000:1.0:1789302716.571110:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 889 00000100:00100000:1.0:1789302716.571120:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+34:25224:x1876219350301696:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302716.571139:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302716.571161:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d51f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02cb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.571235:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.571250:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.571262:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302716.571308:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302716.571313:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302716.571333:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.571371:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d51f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.571389:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 35 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.571398:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:1.0:1789302716.571401:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302716.571406:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7ec00, rc 0) 00000100:00100000:1.0:1789302716.571467:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:25224:x1876219350301696:12345-192.168.203.11@tcp:101:rm.0 Request processed in 349us (532us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302716.571474:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 889 00000100:00100000:2.0:1789302716.580274:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.582561:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301824 00000100:00100000:0.0:1789302716.582576:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301824 00000100:00100000:0.0:1789302716.582659:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 890 00000100:00100000:0.0:1789302716.582670:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+35:25224:x1876219350301824:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302716.582690:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.582718:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d52d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02d2 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582807:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582821:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582833:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582849:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582857:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582864:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582953:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d542 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582963:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d542 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.582970:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d542 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302716.582997:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000004:00200000:0.0:1789302716.583001:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302716.583018:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d542 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583042:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583049:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583057:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583062:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.583067:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583075:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583088:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302716.583091:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.583096:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583103:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d534 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583115:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583122:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583128:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583140:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.583144:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583149:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583157:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xb0a77ca6 (ffff960f03501600) 00010000:00010000:0.0:1789302716.583159:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.583161:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583167:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d53b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xb0a77ca6 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583199:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d52d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02d2 expref: 37 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583211:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb400/0x64fe43095184d542 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02d2 expref: 36 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.583219:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302716.583220:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302716.583223:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb400, rc 0) 00000100:00100000:0.0:1789302716.583294:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25224:x1876219350301824:12345-192.168.203.11@tcp:101:rm.0 Request processed in 626us (3028us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302716.583302:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 890 00000100:00100000:0.0:1789302716.590566:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302716.590713:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350301952 00000100:00100000:3.0:1789302716.590724:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350301952 00000100:00100000:3.0:1789302716.590783:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 891 00000100:00100000:3.0:1789302716.590793:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25224:x1876219350301952:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302716.590816:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302716.590838:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d549 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02d9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.590906:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.590921:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.590933:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.590948:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.590982:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.590990:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591045:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591053:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591061:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591068:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302716.591072:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591080:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591092:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:3.0:1789302716.591095:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302716.591100:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591107:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d550 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591119:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591132:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591138:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591143:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302716.591146:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591152:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591159:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302716.591161:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302716.591164:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591170:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d557 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591203:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d549 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02d9 expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591213:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:3.0:1789302716.591215:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302716.591218:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d549 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02d9 expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.591239:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2800, rc 0) 00000100:00100000:3.0:1789302716.591302:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25224:x1876219350301952:12345-192.168.203.11@tcp:101:rm.0 Request processed in 510us (745us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302716.591308:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 891 00000100:00100000:3.0:1789302716.601019:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302716.601372:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350302080 00000100:00100000:2.0:1789302716.601383:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350302080 00000100:00100000:2.0:1789302716.601456:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 892 00000100:00100000:2.0:1789302716.601467:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25224:x1876219350302080:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302716.601489:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302716.601550:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d55e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02e0 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601627:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601651:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601663:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601678:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601687:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601694:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601754:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601763:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601770:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601787:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302716.601791:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601798:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601810:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:2.0:1789302716.601813:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302716.601818:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601824:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d565 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601842:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601848:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601854:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601859:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302716.601862:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601867:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601886:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f03927900) 00010000:00010000:2.0:1789302716.601888:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302716.601891:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601902:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d56c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601938:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d55e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02e0 expref: 37 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601947:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:2.0:1789302716.601949:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302716.601952:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d55e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02e0 expref: 37 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.601971:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe400, rc 0) 00000100:00100000:2.0:1789302716.602038:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25224:x1876219350302080:12345-192.168.203.11@tcp:101:rm.0 Request processed in 573us (1028us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302716.602044:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 892 00000100:00100000:2.0:1789302716.622798:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.622901:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350302208 00000100:00100000:0.0:1789302716.622911:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350302208 00000100:00100000:0.0:1789302716.622983:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 893 00000100:00100000:0.0:1789302716.622992:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25225:x1876219350302208:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302716.623994:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.624022:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d573 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02e7 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624091:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624106:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624118:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624132:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624287:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624296:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624352:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624360:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624931:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624937:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.624941:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624949:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624960:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302716.624962:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.624966:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624972:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d57a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624984:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624989:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.624995:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.625000:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.625003:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.625027:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.625035:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f03501600) 00010000:00010000:0.0:1789302716.625037:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.625040:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.625046:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d581 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.625086:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d573 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02e7 expref: 37 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.625096:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302716.625098:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.625101:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d573 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02e7 expref: 37 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.625110:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea800, rc 0) 00000100:00100000:0.0:1789302716.628726:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25225:x1876219350302208:12345-192.168.203.11@tcp:101:cp.0 Request processed in 5721us (5923us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302716.628738:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 893 00000100:00100000:0.0:1789302716.637457:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302716.637534:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350302336 00000100:00100000:3.0:1789302716.637545:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350302336 00000100:00100000:3.0:1789302716.637606:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 894 00000100:00100000:3.0:1789302716.637617:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25225:x1876219350302336:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302716.637646:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302716.637666:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d588 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02ee expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.637743:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.637757:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639458:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639494:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639524:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639553:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639611:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639619:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639627:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639644:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302716.639649:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639657:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639669:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:3.0:1789302716.639672:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302716.639677:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639684:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d58f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639696:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639702:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639726:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639732:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302716.639735:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639741:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639750:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302716.639752:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302716.639755:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.639761:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d596 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.641962:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d588 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02ee expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.641982:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:3.0:1789302716.641985:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302716.641990:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d588 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02ee expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.642002:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2800, rc 0) 00000100:00100000:3.0:1789302716.642080:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25225:x1876219350302336:12345-192.168.203.11@tcp:101:cp.0 Request processed in 4464us (4632us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302716.642105:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 894 00000100:00100000:2.0:1789302716.657134:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.659484:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350302592 00000100:00100000:0.0:1789302716.659499:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350302592 00000100:00100000:0.0:1789302716.659561:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 895 00000100:00100000:0.0:1789302716.659578:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25225:x1876219350302592:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302716.659609:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.659853:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d59d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02f5 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.659976:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.659991:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.660002:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.660016:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.660023:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.660031:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302716.662122:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x13:0x0], index 0: 0 00010000:00010000:1.0:1789302716.662324:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d5b2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662349:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d5b2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662361:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d5b2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662403:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662410:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662416:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662421:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302716.662425:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662432:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662442:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:1.0:1789302716.662445:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302716.662450:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.662456:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d5a4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302716.662468:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ea55dbc1 reply state 00000000921c43ce transno 4294967889 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302716.662483:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302716.663165:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d59d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02f5 expref: 39 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.663183:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fe00/0x64fe43095184d5b2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02f5 expref: 38 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.663190:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:1.0:1789302716.663193:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302716.663196:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7fe00, rc 0) 00000100:00100000:1.0:1789302716.663275:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:25225:x1876219350302592:12345-192.168.203.11@tcp:101:cp.0 Request processed in 3705us (6149us total) trans 4294967889 rc 0/0 00000100:00100000:1.0:1789302716.663283:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 895 00010000:00010000:1.0:1789302716.665302:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.665317:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.665324:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.665330:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302716.665335:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.665342:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.665375:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f03501600) 00010000:00010000:1.0:1789302716.665378:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302716.665383:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.665390:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5ab lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302716.667002:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.667069:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350302720 00000100:00100000:0.0:1789302716.667079:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350302720 00000100:00100000:0.0:1789302716.667160:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 225 00000100:00100000:0.0:1789302716.667171:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+8:2347:x1876219350302720:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302716.667188:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.667210:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a02fc expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302716.667254:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x13:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302716.667273:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=553s v=(0 0) 00010000:00010000:0.0:1789302716.667277:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302716.667285:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 9 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.667301:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 9 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.667313:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302716.667367:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350302720:12345-192.168.203.11@tcp:101:cp.0 Request processed in 198us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302716.667374:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 225 00000100:00080000:2.0:1789302716.667607:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302716.672608:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302716.672960:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350302848 00000100:00100000:2.0:1789302716.672971:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350302848 00000100:00100000:2.0:1789302716.673031:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 226 00000100:00100000:2.0:1789302716.673042:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:25225:x1876219350302848:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302716.673070:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302716.673090:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0303 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.673122:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.673131:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302716.673135:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302716.673141:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.673177:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.673192:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 10 pid: 13439 timeout: 653 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302716.673207:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371105280:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302716.673308:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ab25e40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302716.679177:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371105280:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302716.679216:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.679236:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302716.679321:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:25225:x1876219350302848:12345-192.168.203.11@tcp:101:cp.0 Request processed in 6281us (6720us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302716.679330:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 226 00000100:00100000:3.0:1789302716.682650:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302716.683147:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350302976 00000100:00100000:1.0:1789302716.683158:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350302976 00010000:00010000:1.0:1789302716.683603:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 10 pid: 13439 timeout: 653 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302716.683660:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:1.0:1789302716.683671:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350302976:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302716.683687:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302716.683817:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 10 pid: 13439 timeout: 653 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.683833:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 10 pid: 13439 timeout: 653 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.683858:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003501a982/0x64fe43095184d5c0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02fc expref: 10 pid: 13439 timeout: 653 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.683877:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302716.683896:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302716.683906:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.683918:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.683959:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302716.683975:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1876219371105408:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302716.684099:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302716.688037:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302716.688140:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7b480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1876219371105408:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302716.688163:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302716.688218:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350302976:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4548us (5584us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302716.688229:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:0.0:1789302716.688460:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350303232 00000100:00100000:0.0:1789302716.688468:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350303232 00000100:00100000:0.0:1789302716.688544:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 184 00000100:00100000:0.0:1789302716.688555:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ec850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25225:x1876219350303232:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302716.688896:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ec850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:25225:x1876219350303232:12345-192.168.203.11@tcp:35:cp.0 Request processed in 342us (867us total) trans 4294967890 rc 0/0 00000100:00100000:0.0:1789302716.688904:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 184 00000100:00100000:3.0:1789302716.741748:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.741862:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350303360 00000100:00100000:0.0:1789302716.741872:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350303360 00000100:00100000:0.0:1789302716.741938:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 896 00000100:00100000:0.0:1789302716.741949:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25228:x1876219350303360:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302716.741986:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.742007:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d5ce lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a030a expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742079:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742095:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742108:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742123:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742130:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742136:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742219:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5e3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742228:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742235:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302716.742278:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x17:0x0]: returning size 158 00000004:00200000:0.0:1789302716.742288:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302716.742308:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742319:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742327:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742333:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742339:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742345:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.742349:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742355:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742366:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:0.0:1789302716.742369:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.742374:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742380:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d5d5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742392:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742407:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742412:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742416:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302716.742420:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742426:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742433:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f03501100) 00010000:00010000:0.0:1789302716.742436:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302716.742439:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742444:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d5dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742475:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d5ce lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a030a expref: 85 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742488:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5e3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a030a expref: 84 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.742494:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:0.0:1789302716.742497:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302716.742559:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb600, rc 0) 00000100:00100000:0.0:1789302716.742626:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25228:x1876219350303360:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 678us (887us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302716.742645:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 896 00000100:00100000:3.0:1789302716.745269:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302716.745362:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350303488 00000100:00100000:2.0:1789302716.745372:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350303488 00000100:00100000:2.0:1789302716.745443:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 227 00000100:00100000:2.0:1789302716.745453:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350303488:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302716.745470:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302716.745485:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184d5ea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0311 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302716.745645:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371105536:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302716.745885:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ab25e40 going to sleep for 11 seconds 00000100:00080000:3.0:1789302716.749098:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789302716.752055:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb size: 0 -> 158 00002000:00010000:1.0:1789302716.752063:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb mtime: 0 -> 1789302716 00002000:00010000:1.0:1789302716.752066:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb atime: 0 -> 1789302716 00002000:00010000:1.0:1789302716.752069:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x13:0x0] updating lvb ctime: 0 -> 1789302716 00000100:00100000:1.0:1789302716.752094:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371105536:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302716.752128:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184d5ea lrc: 1/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0311 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.752152:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c3390f01/0x64fe43095184d5ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0311 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302716.752178:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbee00, rc 0) 00000100:00100000:1.0:1789302716.752244:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2348:x1876219350303488:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 6793us (6991us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302716.752253:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 227 00000100:00100000:1.0:1789302716.776609:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302716.776796:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350303616 00000100:00100000:3.0:1789302716.776807:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350303616 00000100:00100000:3.0:1789302716.776869:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 897 00000100:00100000:3.0:1789302716.776882:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25228:x1876219350303616:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302716.776903:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302716.776926:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d5f1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0318 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.776996:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d5f8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.777011:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d5f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.777023:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d5f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302716.777058:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3a:0x0]: returning size 16384 00000004:00200000:3.0:1789302716.777062:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302716.777078:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d5f8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.777123:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d5f1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0318 expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.777142:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3000/0x64fe43095184d5f8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0318 expref: 85 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302716.777151:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:3.0:1789302716.777154:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302716.777158:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3000, rc 0) 00000100:00100000:3.0:1789302716.777351:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25228:x1876219350303616:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 471us (751us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302716.777359:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 897 00000100:00100000:2.0:1789302716.785066:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302716.785167:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350303744 00000100:00100000:0.0:1789302716.785177:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350303744 00000100:00100000:0.0:1789302716.785234:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 228 00000100:00100000:0.0:1789302716.785244:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350303744:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302716.785262:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302716.785280:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b43a4659/0x64fe43095184d5ff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a031f expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302716.785337:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371105664:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302716.785415:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb66c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302716.788893:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371105664:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302716.788934:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b43a4659/0x64fe43095184d5ff lrc: 1/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a031f expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.788960:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b43a4659/0x64fe43095184d5ff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a031f expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302716.788973:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebe00, rc 0) 00000100:00100000:0.0:1789302716.789042:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350303744:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3799us (3985us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302716.789051:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 228 00000100:00100000:3.0:1789302716.793642:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302716.794548:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350304000 00000100:00100000:2.0:1789302716.794560:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350304000 00000100:00100000:2.0:1789302716.794647:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 35 00000100:00100000:2.0:1789302716.794658:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25228:x1876219350304000:12345-192.168.203.11@tcp:103:lfs.0 00010000:00010000:2.0:1789302716.794684:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302716.794948:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb600/0x64fe43095184d5e3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a030a expref: 85 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302716.794969:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:2.0:1789302716.794975:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302716.795047:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25228:x1876219350304000:12345-192.168.203.11@tcp:103:lfs.0 Request processed in 391us (1426us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302716.795072:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 35 00000100:00100000:3.0:1789302716.807278:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302716.807374:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350304128 00000100:00100000:1.0:1789302716.807385:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350304128 00000100:00100000:1.0:1789302716.807451:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 898 00000100:00100000:1.0:1789302716.807462:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25228:x1876219350304128:12345-192.168.203.11@tcp:58:lfs.0 00000100:00100000:1.0:1789302716.807647:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25228:x1876219350304128:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 186us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302716.807654:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 898 00000040:00080000:1.0:1789302717.070410:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302717.070419:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302717.071108:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:1.0:1789302717.071119:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302717.071205:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302717.865449:0:18666:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302717.865464:0:18666:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302717.867230:0:18666:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 22 count 4 00000040:00080000:0.0:1789302717.867239:0:18666:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302717.986101:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302717.986184:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350304256 00000100:00100000:2.0:1789302717.986194:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350304256 00000100:00100000:2.0:1789302717.986269:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 899 00000100:00100000:2.0:1789302717.986418:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25238:x1876219350304256:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302717.986439:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302717.986462:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d606 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0326 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986557:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986573:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986584:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986600:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986608:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986616:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986718:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d61b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986745:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d61b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986753:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d61b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302717.986786:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x17:0x0]: returning size 158 00000004:00200000:2.0:1789302717.986791:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302717.986807:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d61b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986819:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d61b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986826:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986832:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986839:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986844:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302717.986848:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986855:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986865:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0x0 (ffff960f03501e00) 00010000:00010000:2.0:1789302717.986874:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302717.986880:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986887:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d60d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986905:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986911:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986916:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986921:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbee00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302717.986924:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986930:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986937:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3a:0x0].0xbc70ccd9 (ffff960f03927900) 00010000:00010000:2.0:1789302717.986940:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302717.986943:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986948:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d614 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3a:0x0].0xbc70ccd9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986978:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d606 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3a:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0326 expref: 86 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.986999:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d61b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0326 expref: 85 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302717.987007:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:2.0:1789302717.987010:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302717.987012:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbec00, rc 0) 00000100:00100000:2.0:1789302717.987068:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25238:x1876219350304256:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 790us (975us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302717.987076:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 899 00000100:00100000:3.0:1789302717.997762:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.006170:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350304384 00000100:00100000:1.0:1789302718.006185:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350304384 00000100:00100000:1.0:1789302718.006271:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00100000:1.0:1789302718.006293:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350304384:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302718.006312:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302718.006330:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d622 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a032d expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302718.006417:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371105792:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302718.006526:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1d80 going to sleep for 11 seconds 00000100:00080000:3.0:1789302718.008868:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302718.022683:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371105792:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302718.022794:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d622 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a032d expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.022820:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d622 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a032d expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.022832:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7e000, rc 0) 00000100:00100000:1.0:1789302718.022899:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2347:x1876219350304384:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 16617us (25145us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302718.022909:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00100000:1.0:1789302718.035573:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302718.037775:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350304512 00000100:00100000:0.0:1789302718.037790:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350304512 00000100:00100000:0.0:1789302718.037843:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 230 00000100:00100000:0.0:1789302718.037853:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350304512:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302718.037869:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302718.037886:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000005f0fe3e/0x64fe43095184d629 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0334 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302718.037947:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371105920:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302718.038032:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302718.052270:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371105920:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302718.052308:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000005f0fe3e/0x64fe43095184d629 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0334 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.052331:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000005f0fe3e/0x64fe43095184d629 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0334 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.052342:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:3.0:1789302718.052400:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2346:x1876219350304512:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 14548us (16834us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302718.052408:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 230 00000100:00100000:0.0:1789302718.077089:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302718.077176:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350304768 00000100:00100000:3.0:1789302718.077185:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350304768 00000100:00100000:3.0:1789302718.077254:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 900 00000100:00100000:3.0:1789302718.077264:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25238:x1876219350304768:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302718.077609:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].22, 136 off11184 00000100:00100000:3.0:1789302718.077699:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25238:x1876219350304768:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 436us (619us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302718.077709:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 900 00000100:00080000:2.0:1789302718.079082:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302718.092708:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302718.092874:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302718.092880:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302718.092917:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 5 00000040:00080000:0.0:1789302718.092924:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302718.093193:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371106048:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302718.093286:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302718.095692:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371106048:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302718.102402:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302718.102490:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350304896 00000100:00100000:0.0:1789302718.102530:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350304896 00000100:00100000:0.0:1789302718.102605:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 901 00000100:00100000:0.0:1789302718.102617:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:25240:x1876219350304896:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302718.102663:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x17:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302718.102761:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:25240:x1876219350304896:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 146us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302718.102785:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 901 00000100:00080000:2.0:1789302718.102851:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302718.116608:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302718.116686:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350305152 00000100:00100000:2.0:1789302718.116696:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350305152 00000100:00100000:2.0:1789302718.116754:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 902 00000100:00100000:2.0:1789302718.116764:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:25240:x1876219350305152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302718.116785:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302718.116806:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d630 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a033b expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.116905:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d637 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.116920:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d637 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.116932:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d637 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302718.116959:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x17:0x0]: returning size 158 00010000:00010000:2.0:1789302718.117085:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d630 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a033b expref: 42 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.117118:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d637 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a033b expref: 41 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.117128:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:2.0:1789302718.117131:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302718.117135:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe400, rc 0) 00000100:00100000:2.0:1789302718.117204:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25240:x1876219350305152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 441us (604us total) trans 4294967891 rc 0/0 00000100:00100000:2.0:1789302718.117212:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 902 00000100:00100000:1.0:1789302718.126042:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302718.126104:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350305280 00000100:00100000:3.0:1789302718.126115:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350305280 00000100:00100000:3.0:1789302718.126183:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 36 00000100:00100000:3.0:1789302718.126194:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2345:x1876219350305280:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302718.126210:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302718.126234:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d637 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a033b expref: 41 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.126252:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:3.0:1789302718.126257:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302718.126324:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b2050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:2345:x1876219350305280:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 132us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302718.126331:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 36 00000100:00100000:0.0:1789302718.139191:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.139287:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350305408 00000100:00100000:1.0:1789302718.139296:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350305408 00000100:00100000:1.0:1789302718.139426:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 231 00000100:00100000:1.0:1789302718.139436:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350305408:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302718.139480:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.139552:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.139561:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x13:0x0].0x0 since=2s v=(0 0) 00010000:00010000:1.0:1789302718.139565:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302718.139570:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.139604:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 9 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.139618:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 9 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302718.139645:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371106176:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302718.140074:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1d80 going to sleep for 11 seconds 00000100:00080000:2.0:1789302718.141449:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302718.144115:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371106176:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302718.144144:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302718.146154:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302718.149032:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350305536 00000100:00100000:2.0:1789302718.149048:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350305536 00002000:00100000:2.0:1789302718.149110:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x13:0x0] (0->157) req@ffff960e02908700 x1876219350305536/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:294/0 lens 648/0 e 0 to 0 dl 1789302729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302718.149122:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02908700 with x1876219350305536 ext(0->157) 00010000:00010000:2.0:1789302718.149131:0:6700:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 10 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.149149:0:6700:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 10 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302718.149158:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960e02908700 00000100:00100000:2.0:1789302718.149268:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 5 00000100:00100000:2.0:1789302718.149279:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350305536:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:2.0:1789302718.151686:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350305536:12345-192.168.203.11@tcp:4:cp.0 Request processed in 2399us (5529us total) trans 4294967344 rc 0/0 00000100:00100000:2.0:1789302718.151708:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 5 00002000:00100000:2.0:1789302718.151721:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x13:0x0] (0->157) req@ffff960e02908700 x1876219350305536/t4294967344(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:294/0 lens 648/448 e 0 to 0 dl 1789302729 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302718.151732:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02908700 with x1876219350305536 ext(0->157) 00010000:00010000:2.0:1789302718.151741:0:6700:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 12 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.151761:0:6700:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 12 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302718.151842:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960e02908700 00000100:00100000:2.0:1789302718.172117:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.172224:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350305664 00000100:00100000:1.0:1789302718.172235:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350305664 00010000:00010000:1.0:1789302718.172443:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 12 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302718.172572:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 278 00000100:00100000:1.0:1789302718.172583:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350305664:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302718.172597:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302718.172617:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 12 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.172642:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 12 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.172664:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe43095184d5c7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0303 expref: 12 pid: 6697 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.172681:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302718.172685:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302718.172704:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.172714:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.172737:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302718.172795:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350305664:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 214us (688us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302718.172803:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 278 00010000:00010000:2.0:1789302718.175905:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.175924:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.175962:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.175991:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.175999:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.176007:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7f800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302718.176011:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.176033:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.176054:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302718.176061:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000038f75c10/0x64fe43095184d645 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302718.176168:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350305408:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 36729us (36982us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302718.176179:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 231 00000100:00100000:2.0:1789302718.188339:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.188438:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350305920 00000100:00100000:1.0:1789302718.188448:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350305920 00000100:00100000:1.0:1789302718.188545:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 903 00000100:00100000:1.0:1789302718.188555:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:25240:x1876219350305920:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302718.188671:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:25240:x1876219350305920:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 117us (339us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302718.188679:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 903 00000100:00100000:3.0:1789302718.195847:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.196112:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350306048 00000100:00100000:1.0:1789302718.196122:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350306048 00000100:00100000:1.0:1789302718.196185:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 232 00000100:00100000:1.0:1789302718.196194:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350306048:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302718.196211:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302718.196234:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004fef09bd/0x64fe43095184d64c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0342 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302718.196259:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x13:0x0] initial LVB size: 158, mtime: 0x6aa697bc, atime: 0x0, ctime: 0x6aa697bc, blocks: 0x1 00010000:00010000:1.0:1789302718.196278:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=555s v=(0 0) 00010000:00010000:1.0:1789302718.196293:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302718.196302:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004fef09bd/0x64fe43095184d64c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0342 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.196318:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004fef09bd/0x64fe43095184d64c lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0342 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.196330:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7e400, rc 0) 00000100:00100000:1.0:1789302718.196578:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350306048:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 384us (739us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302718.196599:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 232 00000100:00100000:3.0:1789302718.201967:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.202306:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350306304 00000100:00100000:1.0:1789302718.202317:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350306304 00000100:00100000:1.0:1789302718.202491:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 904 00000100:00100000:1.0:1789302718.202540:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:25240:x1876219350306304:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302718.202654:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:25240:x1876219350306304:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 153us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302718.202661:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 904 00000100:00100000:2.0:1789302718.210495:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302718.214883:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350306432 00000100:00100000:0.0:1789302718.214898:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350306432 00002000:00100000:0.0:1789302718.214956:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x13:0x0] (0->4095) req@ffff960e02322300 x1876219350306432/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:294/0 lens 488/0 e 0 to 0 dl 1789302729 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302718.214968:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02322300 with x1876219350306432 ext(0->4095) 00002000:00010000:0.0:1789302718.214974:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e02322300 00000100:00100000:0.0:1789302718.215075:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 88 00000100:00100000:0.0:1789302718.215087:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350306432:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302718.215296:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350306432:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 210us (5038us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302718.215304:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 88 00002000:00100000:0.0:1789302718.215311:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x13:0x0] (0->4095) req@ffff960e02322300 x1876219350306432/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:294/0 lens 488/4536 e 0 to 0 dl 1789302729 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302718.215320:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02322300 with x1876219350306432 ext(0->4095) 00002000:00010000:0.0:1789302718.215323:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e02322300 00000100:00100000:2.0:1789302718.222338:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302718.224823:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350306560 00000100:00100000:3.0:1789302718.224838:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350306560 00000100:00100000:3.0:1789302718.224907:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 905 00000100:00100000:3.0:1789302718.224917:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:25240:x1876219350306560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302718.224937:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302718.224958:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d653 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0349 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.224991:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.225006:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.225016:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.227609:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d653 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.227652:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 41 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.227661:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:3.0:1789302718.227665:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302718.227668:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2800, rc 0) 00000100:00100000:3.0:1789302718.227745:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25240:x1876219350306560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2828us (5414us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302718.227767:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 905 00000100:00100000:3.0:1789302718.237357:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302718.237902:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350306688 00000100:00100000:0.0:1789302718.237912:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350306688 00000100:00100000:0.0:1789302718.240926:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 906 00000100:00100000:0.0:1789302718.240947:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25240:x1876219350306688:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302718.240982:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x17:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302718.241090:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25240:x1876219350306688:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 151us (3741us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302718.241105:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 906 00000100:00100000:3.0:1789302718.244693:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.245945:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350306816 00000100:00100000:1.0:1789302718.245958:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350306816 00000100:00100000:1.0:1789302718.246023:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 233 00000100:00100000:1.0:1789302718.246035:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350306816:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302718.246182:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246200:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302718.246205:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302718.246211:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246226:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246257:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246338:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246347:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246353:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302718.246358:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246368:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246384:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302718.246391:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000336579d8/0x64fe43095184d661 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302718.246460:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350306816:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 427us (1775us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302718.246468:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 233 00000100:00100000:1.0:1789302718.251042:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302718.251183:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350307072 00000100:00100000:3.0:1789302718.251193:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350307072 00000100:00100000:3.0:1789302718.251247:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 907 00000100:00100000:3.0:1789302718.251257:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25240:x1876219350307072:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302718.251344:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.251361:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 41 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.251370:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.251403:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 41 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.251418:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 41 pid: 5845 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302718.251433:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371106304:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302718.251548:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33589b40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302718.254986:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302718.255133:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371106304:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302718.255171:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302718.257277:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350307200 00000100:00100000:0.0:1789302718.257294:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350307200 00010000:00010000:0.0:1789302718.257342:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 42 pid: 5845 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302718.257392:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 279 00000100:00100000:0.0:1789302718.257402:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:2347:x1876219350307200:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302718.257425:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302718.257433:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 42 pid: 5845 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302718.257451:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 42 pid: 5845 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302718.257477:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d65a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0349 expref: 42 pid: 5845 timeout: 655 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302718.257493:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:0.0:1789302718.257534:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302718.257548:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302718.257559:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302718.257614:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302718.257660:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.257682:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302718.257688:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2347:x1876219350307200:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 285us (2708us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302718.257701:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 279 00010000:00010000:3.0:1789302718.257883:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.257891:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.257898:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.257904:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302718.257908:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.257916:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.257926:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:3.0:1789302718.257928:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302718.257932:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302718.257938:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d668 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x17:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302718.258060:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25240:x1876219350307072:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 6804us (7027us total) trans 4294967892 rc 0/0 00000100:00100000:3.0:1789302718.258068:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 907 00000040:00080000:0.0:1789302718.258342:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302718.258346:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302718.259194:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:0.0:1789302718.259204:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302718.261822:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302718.261938:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350307456 00000100:00100000:3.0:1789302718.261948:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350307456 00000100:00100000:3.0:1789302718.262007:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 185 00000100:00100000:3.0:1789302718.262018:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25240:x1876219350307456:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302718.262566:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25240:x1876219350307456:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 548us (749us total) trans 4294967893 rc 0/0 00000100:00100000:3.0:1789302718.262584:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 185 00000040:00080000:0.0:1789302719.075208:0:18716:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302719.075217:0:18716:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302719.075266:0:18716:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 5 00000040:00080000:0.0:1789302719.075271:0:18716:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302719.118199:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302719.184134:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302719.185604:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350307584 00000100:00100000:0.0:1789302719.185633:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350307584 00000100:00100000:0.0:1789302719.185698:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 234 00000100:00100000:0.0:1789302719.185710:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+10:2345:x1876219350307584:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302719.185728:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302719.185747:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007c837df7/0x64fe43095184d66f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0350 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302719.185897:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789302719.185904:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302719.185914:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007c837df7/0x64fe43095184d66f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0350 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302719.185954:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007c837df7/0x64fe43095184d66f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0350 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302719.185967:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb600, rc 0) 00000100:00100000:0.0:1789302719.186028:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350307584:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 320us (1902us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302719.186035:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 234 00000100:00100000:2.0:1789302719.197148:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302719.197288:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350307840 00000100:00100000:1.0:1789302719.197299:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350307840 00000100:00100000:1.0:1789302719.197366:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 37 00000100:00100000:1.0:1789302719.197377:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25249:x1876219350307840:12345-192.168.203.11@tcp:103:lfs.0 00010000:00010000:1.0:1789302719.197404:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302719.197427:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d61b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x17:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0326 expref: 85 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302719.197444:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x17:0x0].0x0 (ffff960f10d24400) 00010000:00010000:1.0:1789302719.197450:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302719.198367:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25249:x1876219350307840:12345-192.168.203.11@tcp:103:lfs.0 Request processed in 986us (1222us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302719.198377:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 37 00000100:00100000:0.0:1789302719.204364:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302719.205750:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350307968 00000100:00100000:1.0:1789302719.205763:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350307968 00000100:00100000:1.0:1789302719.205842:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 908 00000100:00100000:1.0:1789302719.205853:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25249:x1876219350307968:12345-192.168.203.11@tcp:58:lfs.0 00000100:00100000:1.0:1789302719.205984:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25249:x1876219350307968:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 132us (1629us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302719.205990:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 908 00000040:00080000:0.0:1789302719.320283:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302719.320293:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302719.320456:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 3 00000040:00080000:0.0:1789302719.320462:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302719.500847:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371106432:0@lo:400:kworker.0 00000100:00100000:1.0:1789302719.500919:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302719.501109:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371106688:0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.501140:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302719.501162:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789302719.501169:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371106560:0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.501189:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302719.501204:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371107072:0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.501227:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302719.501258:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371106688 00000100:00100000:0.0:1789302719.501265:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371106688 00000100:00100000:1.0:1789302719.501422:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371106560 00000100:00100000:1.0:1789302719.501427:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371106560 00000100:00100000:1.0:1789302719.501472:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 353 00000100:00100000:1.0:1789302719.501480:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371106560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302719.501552:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371106560:12345-0@lo:400:kworker.0 Request processed in 72us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302719.501559:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 353 00000100:00100000:1.0:1789302719.501581:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107072 00000100:00100000:1.0:1789302719.501584:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107072 00000100:00100000:1.0:1789302719.501627:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:1.0:1789302719.501634:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371107072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302719.501665:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371107072:12345-0@lo:400:kworker.0 Request processed in 32us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302719.501672:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00080000:1.0:1789302719.501700:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302719.501730:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371106560:0@lo:400:kworker.0 00000100:00100000:1.0:1789302719.501741:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c93800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371107072:0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.501892:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:0.0:1789302719.501899:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371106688:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.501931:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371106688:12345-0@lo:400:kworker.0 Request processed in 33us (790us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302719.501945:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1789302719.502060:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371106816:0@lo:400:kworker.0 00000100:00100000:3.0:1789302719.502090:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302719.502123:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302719.502128:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371106944:0@lo:400:kworker.0 00000100:00100000:3.0:1789302719.502148:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302719.502829:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371106816 00000100:00100000:0.0:1789302719.502836:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371106816 00000100:00100000:0.0:1789302719.502866:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1789302719.502873:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371106816:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.502920:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371106816:12345-0@lo:400:kworker.0 Request processed in 48us (829us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302719.502926:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 337 00000100:00100000:0.0:1789302719.502943:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371106944 00000100:00100000:0.0:1789302719.502946:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371106944 00000100:00100000:0.0:1789302719.502969:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:0.0:1789302719.502976:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371106944:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.503003:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371106944:12345-0@lo:400:kworker.0 Request processed in 29us (854us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302719.503008:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00100000:0.0:1789302719.503039:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c92a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371106944:0@lo:400:kworker.0 00000100:00100000:0.0:1789302719.503049:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371106816:0@lo:400:kworker.0 00000100:00100000:1.0:1789302719.503112:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371106688:0@lo:400:kworker.0 00000100:00080000:1.0:1789302719.503132:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302719.503136:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302719.503211:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371106432 00000100:00100000:2.0:1789302719.503216:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371106432 00000100:00100000:2.0:1789302719.503252:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 542 00000100:00100000:2.0:1789302719.503259:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371106432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302719.503315:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371106432:12345-0@lo:400:kworker.0 Request processed in 57us (2396us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.503322:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 542 00000100:00100000:1.0:1789302719.503491:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371106432:0@lo:400:kworker.0 00000100:00100000:2.0:1789302719.759126:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302719.759158:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302719.759179:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302719.759203:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302719.759223:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302719.759241:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302719.759264:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302719.759291:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308480 00000100:00100000:2.0:1789302719.759300:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308480 00000100:00100000:2.0:1789302719.759350:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308736 00000100:00100000:2.0:1789302719.759353:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308736 00000100:00100000:2.0:1789302719.759370:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308352 00000100:00100000:2.0:1789302719.759373:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308352 00000100:00100000:2.0:1789302719.759383:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308864 00000100:00100000:2.0:1789302719.759386:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308864 00000100:00100000:2.0:1789302719.759402:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:2.0:1789302719.759410:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350308480:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302719.759481:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350308480:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 71us (321us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.759488:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:2.0:1789302719.759529:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:2.0:1789302719.759536:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350308736:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302719.759562:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350308736:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.759568:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:2.0:1789302719.759579:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:2.0:1789302719.759585:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350308352:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302719.759606:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350308352:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.759625:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:2.0:1789302719.759636:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:2.0:1789302719.759642:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350308864:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302719.759663:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350308864:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.759669:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:2.0:1789302719.759688:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308096 00000100:00100000:2.0:1789302719.759691:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308096 00000100:00100000:2.0:1789302719.759726:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 543 00000100:00100000:2.0:1789302719.759732:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350308096:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302719.759755:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350308096:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (551us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.759760:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 543 00000100:00100000:2.0:1789302719.759786:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308224 00000100:00100000:2.0:1789302719.759790:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308224 00000100:00100000:2.0:1789302719.759834:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308608 00000100:00100000:2.0:1789302719.759837:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308608 00000100:00100000:2.0:1789302719.759856:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 356 00000100:00100000:2.0:1789302719.759862:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:2345:x1876219350308224:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302719.759893:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:2345:x1876219350308224:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (776us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.759900:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 356 00000100:00100000:2.0:1789302719.759911:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:2.0:1789302719.759917:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:2345:x1876219350308608:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302719.759942:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:2345:x1876219350308608:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (700us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302719.759948:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00080000:1.0:1789302719.765251:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302720.012817:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371107200:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302720.012881:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302720.013059:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107200 00000100:00100000:3.0:1789302720.013069:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107200 00000100:00100000:3.0:1789302720.013119:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:3.0:1789302720.013128:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371107200:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302720.013161:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 69664768 pending 0 free 3851419648 avail 3849322496 00000100:00100000:3.0:1789302720.013213:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371107200:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302720.013221:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:2.0:1789302720.022415:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371107200:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789302720.075721:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967890 is committed 00000001:00080000:0.0:1789302720.075737:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967888 is committed 00000020:00080000:0.0:1789302720.075754:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 14 batchid updated 00000001:00080000:0.0:1789302720.075770:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967891 is committed 00000001:00080000:0.0:1789302720.075775:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967893 is committed 00000020:00080000:0.0:1789302720.075791:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 14 00000020:00080000:0.0:1789302720.075800:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 14 00000001:00080000:0.0:1789302720.112564:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967348 is committed 00010000:00010000:1.0:1789302720.141281:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302720.193740:0:18769:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302720.193755:0:18769:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302720.193810:0:18769:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 23 count 3 00000040:00080000:1.0:1789302720.193817:0:18769:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302720.524894:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302720.524923:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371107328:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302720.524990:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302720.525283:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107328 00000100:00100000:3.0:1789302720.525291:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107328 00000100:00100000:3.0:1789302720.525352:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 153 00000100:00100000:3.0:1789302720.525363:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371107328:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302720.525396:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 74776576 pending 1703936 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302720.525451:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371107328:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (461us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302720.525471:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 153 00000100:00100000:1.0:1789302720.525564:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371107328:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302721.164869:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302722.192450:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789302722.226903:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967344 is committed 00010000:00010000:1.0:1789302723.213971:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302724.240605:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302724.626869:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371107584:0@lo:400:kworker.0 00000100:00100000:3.0:1789302724.626964:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302724.627037:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371108096:0@lo:400:kworker.0 00000100:00100000:3.0:1789302724.627069:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302724.627101:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302724.627108:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371107712:0@lo:400:kworker.0 00000100:00100000:0.0:1789302724.627134:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371107840:0@lo:400:kworker.0 00000100:00100000:3.0:1789302724.627135:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302724.627178:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302724.627188:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107584 00000100:00100000:3.0:1789302724.627198:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107584 00000100:00100000:0.0:1789302724.627210:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1789302724.627218:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371107456:0@lo:400:kworker.0 00000100:00100000:0.0:1789302724.627253:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302724.627278:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371107968:0@lo:400:kworker.0 00000100:00100000:0.0:1789302724.627311:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302724.627915:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371108096 00000100:00100000:2.0:1789302724.627923:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371108096 00000100:00080000:1.0:1789302724.627973:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302724.627985:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107840 00000100:00100000:2.0:1789302724.627990:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107840 00000100:00080000:1.0:1789302724.628023:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302724.628049:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1789302724.628062:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371108096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302724.628148:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371108096:12345-0@lo:400:kworker.0 Request processed in 89us (1076us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.628156:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1789302724.628177:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 343 00000100:00100000:2.0:1789302724.628185:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371107840:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302724.628235:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371107840:12345-0@lo:400:kworker.0 Request processed in 51us (1057us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.628241:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 343 00000100:00100000:0.0:1789302724.628445:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371107840:0@lo:400:kworker.0 00000100:00100000:0.0:1789302724.628612:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107456 00000100:00100000:0.0:1789302724.628618:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107456 00000100:00100000:0.0:1789302724.628646:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 544 00000100:00100000:0.0:1789302724.628653:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371107456:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302724.628695:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371107456:12345-0@lo:400:kworker.0 Request processed in 43us (1441us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302724.628702:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 544 00000100:00100000:3.0:1789302724.628787:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107712 00000100:00100000:0.0:1789302724.628788:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371107968 00000100:00100000:0.0:1789302724.628795:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107968 00000100:00100000:3.0:1789302724.628796:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371107712 00000100:00100000:0.0:1789302724.628833:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:3.0:1789302724.628840:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 359 00000100:00100000:0.0:1789302724.628844:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371107584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302724.628855:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371107968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302724.628882:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371107584:12345-0@lo:400:kworker.0 Request processed in 42us (1918us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302724.628889:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1789302724.628906:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789302724.628911:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371107968:12345-0@lo:400:kworker.0 Request processed in 59us (1607us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302724.628917:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371107712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302724.628925:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 359 00000100:00100000:0.0:1789302724.628952:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371107712:12345-0@lo:400:kworker.0 Request processed in 39us (1817us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302724.628965:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789302724.628975:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371107712:0@lo:400:kworker.0 00000100:00080000:0.0:1789302724.628990:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302724.628995:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371107584:0@lo:400:kworker.0 00000100:00100000:3.0:1789302724.629012:0:3302:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967348 gen 2 00000004:00080000:3.0:1789302724.629019:0:3302:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f7c9c00, transno 4294967348 00000100:00100000:0.0:1789302724.629019:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371107456:0@lo:400:kworker.0 00000100:00100000:0.0:1789302724.629035:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371107968:0@lo:400:kworker.0 00000100:00100000:3.0:1789302724.629037:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371108096:0@lo:400:kworker.0 00000100:00100000:2.0:1789302724.878968:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302724.879013:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302724.879035:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302724.879061:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302724.879080:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302724.879097:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302724.879133:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302724.879162:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350309376 00000100:00100000:2.0:1789302724.879171:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350309376 00000100:00100000:2.0:1789302724.879224:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350309632 00000100:00100000:2.0:1789302724.879228:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350309632 00000100:00100000:2.0:1789302724.879247:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350309248 00000100:00100000:2.0:1789302724.879249:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350309248 00000100:00100000:2.0:1789302724.879262:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350309760 00000100:00100000:2.0:1789302724.879264:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350309760 00000100:00100000:2.0:1789302724.879281:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 344 00000100:00100000:2.0:1789302724.879291:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2346:x1876219350309376:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302724.879360:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2346:x1876219350309376:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 70us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.879367:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 344 00000100:00100000:2.0:1789302724.879384:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:2.0:1789302724.879391:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350309632:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302724.879430:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350309632:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.879449:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:2.0:1789302724.879462:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 346 00000100:00100000:2.0:1789302724.879468:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350309248:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302724.879495:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350309248:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (414us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.879527:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 346 00000100:00100000:2.0:1789302724.879541:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:2.0:1789302724.879548:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350309760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302724.879571:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350309760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.879587:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:2.0:1789302724.879606:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350308992 00000100:00100000:2.0:1789302724.879610:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350308992 00000100:00100000:2.0:1789302724.879636:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 545 00000100:00100000:2.0:1789302724.879643:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350308992:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302724.879666:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350308992:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (604us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.879672:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 545 00000100:00100000:2.0:1789302724.879702:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350309120 00000100:00100000:2.0:1789302724.879705:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350309120 00000100:00100000:2.0:1789302724.879732:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350309504 00000100:00100000:2.0:1789302724.879734:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350309504 00000100:00100000:2.0:1789302724.879754:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 361 00000100:00100000:2.0:1789302724.879760:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:2346:x1876219350309120:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302724.879798:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:2346:x1876219350309120:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (839us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.879804:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 361 00000100:00100000:2.0:1789302724.879816:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:2.0:1789302724.879822:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2346:x1876219350309504:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302724.879847:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2346:x1876219350309504:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (749us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302724.879853:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00080000:2.0:1789302724.879874:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302724.879878:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302724.879881:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302725.133010:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371108224:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302725.133078:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302725.133149:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371108224 00000100:00100000:0.0:1789302725.133157:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371108224 00000100:00100000:0.0:1789302725.133198:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789302725.133205:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371108224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302725.133234:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 1703936 granted 69664768 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302725.133274:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371108224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302725.133280:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:0.0:1789302725.133335:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371108224:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302725.260965:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302725.661053:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371108352:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302725.661150:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302725.661220:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371108352 00000100:00100000:3.0:1789302725.661228:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371108352 00000100:00100000:3.0:1789302725.661272:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 155 00000100:00100000:3.0:1789302725.661281:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371108352:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302725.661314:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 73072640 pending 0 free 3857711104 avail 3855613952 00000100:00100000:3.0:1789302725.661359:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371108352:12345-0@lo:13:osp-pre-0-0.0 Request processed in 79us (209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302725.661367:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 155 00000100:00100000:3.0:1789302725.661409:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371108352:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302725.955497:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302725.956380:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350309888 00000100:00100000:3.0:1789302725.956392:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350309888 00000100:00100000:3.0:1789302725.956459:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 909 00000100:00100000:3.0:1789302725.956470:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:25221:x1876219350309888:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1789302725.956615:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:25221:x1876219350309888:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 147us (1126us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302725.956622:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 909 00010000:00010000:1.0:1789302726.288295:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302727.310177:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1789302728.082424:0:18969:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12e: Check forbidden restore ========== 08:32:06 (1789302726) 00010000:00010000:0.0:1789302728.334929:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302728.573760:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302728.573914:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350310016 00000100:00100000:0.0:1789302728.573924:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350310016 00000100:00100000:0.0:1789302728.574021:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 910 00000100:00100000:0.0:1789302728.574042:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:25861:x1876219350310016:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1789302728.574152:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:25861:x1876219350310016:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 114us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302728.574180:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 910 00000100:00080000:2.0:1789302728.574810:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302728.617904:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302728.618169:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350310144 00000100:00100000:1.0:1789302728.618180:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350310144 00000100:00100000:1.0:1789302728.618299:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 911 00000100:00100000:1.0:1789302728.618310:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25862:x1876219350310144:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302728.618330:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302728.618352:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d676 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0357 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618416:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618430:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618442:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618472:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618495:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618532:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618754:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618762:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618769:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618775:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302728.618780:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618787:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618798:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:1.0:1789302728.618802:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302728.618807:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618814:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d67d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618826:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618839:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618845:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618851:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7f800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302728.618854:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618861:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618869:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff960f10d24300) 00010000:00010000:1.0:1789302728.618873:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302728.618876:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618882:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d684 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618913:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d676 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0357 expref: 84 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.618924:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:1.0:1789302728.618926:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00080000:2.0:1789302728.619290:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302728.619378:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d676 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0357 expref: 84 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.619392:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7e000, rc 0) 00000100:00100000:1.0:1789302728.619542:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25862:x1876219350310144:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1194us (1608us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302728.619559:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 911 00000100:00100000:1.0:1789302728.635895:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302728.636136:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350310400 00000100:00100000:3.0:1789302728.636146:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350310400 00000100:00100000:3.0:1789302728.636210:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 912 00000100:00100000:3.0:1789302728.636220:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25862:x1876219350310400:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:3.0:1789302728.636274:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302728.636299:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d4e7 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02b6 expref: 83 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.636337:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302728.636343:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302728.636775:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.636790:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.636800:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.636832:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 36 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.636846:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 36 pid: 5844 timeout: 665 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302728.636860:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371108480:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302728.636988:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33589540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302728.646052:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302728.646648:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371108480:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302728.646679:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302728.648629:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350310528 00000100:00100000:1.0:1789302728.648643:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350310528 00010000:00010000:1.0:1789302728.648690:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 37 pid: 5844 timeout: 665 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302728.648735:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 280 00000100:00100000:1.0:1789302728.648744:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:2348:x1876219350310528:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302728.648759:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302728.648765:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 37 pid: 5844 timeout: 665 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.648778:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 37 pid: 5844 timeout: 665 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.648800:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d526 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a02cb expref: 37 pid: 5844 timeout: 665 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.648816:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:1.0:1789302728.648822:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.648831:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.648856:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.648899:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302728.649182:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.649190:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.649215:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.649237:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.649244:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302728.650559:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 16384 00010000:00010000:3.0:1789302728.651148:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.651160:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.651166:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.651172:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302728.651177:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.651185:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.651199:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302728.651202:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.651206:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.651211:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d68b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302728.651246:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f0786525 reply state 00000000d398a57c transno 4294967894 ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302728.651254:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302728.651429:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350310528:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 2680us (5380us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302728.651439:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 280 00000100:00100000:3.0:1789302728.651769:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25862:x1876219350310400:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 15549us (15882us total) trans 4294967894 rc 0/0 00000100:00100000:3.0:1789302728.651777:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 912 00010000:00010000:2.0:1789302728.662701:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.662717:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.662735:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.662740:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302728.662745:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.662771:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.662788:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff960f0cdf2c00) 00010000:00010000:2.0:1789302728.662791:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302728.662797:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.662803:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d692 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302728.669740:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.672165:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350310656 00000100:00100000:2.0:1789302728.672182:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350310656 00000100:00100000:2.0:1789302728.672255:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 913 00000100:00100000:2.0:1789302728.672270:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+83:25863:x1876219350310656:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302728.672292:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302728.672314:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d699 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a035e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.672373:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d6a0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.672387:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d6a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.672397:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d6a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302728.672427:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302728.672442:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302728.672461:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d6a0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.672482:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d699 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a035e expref: 85 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.672518:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d6a0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a035e expref: 84 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.672527:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302728.672530:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302728.672533:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbee00, rc 0) 00000100:00100000:2.0:1789302728.672606:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25863:x1876219350310656:12345-192.168.203.11@tcp:101:bash.0 Request processed in 337us (2877us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302728.672613:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 913 00000100:00100000:0.0:1789302728.675293:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302728.675433:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350310784 00000100:00100000:1.0:1789302728.675443:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350310784 00000100:00100000:1.0:1789302728.675499:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 914 00000100:00100000:1.0:1789302728.675540:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+84:25863:x1876219350310784:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302728.675568:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302728.675588:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d6a7 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0365 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675664:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675677:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675687:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675701:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675708:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675732:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675871:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d6bc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675879:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d6bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.675886:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d6bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302728.675926:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 16384 00000004:00200000:1.0:1789302728.676275:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302728.676304:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d6bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676316:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676321:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676327:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676333:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7f800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302728.676336:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676343:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676363:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:1.0:1789302728.676366:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302728.676372:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676378:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d6ae lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676392:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676397:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676402:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676407:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302728.676409:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676415:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676422:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff960f10d24300) 00010000:00010000:1.0:1789302728.676424:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302728.676427:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676433:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d6b5 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676477:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d6a7 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0365 expref: 86 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676490:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ec00/0x64fe43095184d6bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0365 expref: 85 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.676496:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302728.676499:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302728.676533:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7ec00, rc 0) 00000100:00100000:1.0:1789302728.676601:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25863:x1876219350310784:12345-192.168.203.11@tcp:101:bash.0 Request processed in 1062us (1316us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302728.676609:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 914 00000100:00100000:1.0:1789302728.681778:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302728.682063:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350310912 00000100:00100000:3.0:1789302728.682073:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350310912 00000100:00100000:3.0:1789302728.682130:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 915 00000100:00100000:3.0:1789302728.682140:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25863:x1876219350310912:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302728.682160:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302728.682179:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d6c3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a036c expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682253:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682279:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682292:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682355:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682364:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682388:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682450:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682458:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682465:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682470:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302728.682474:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682482:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682493:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302728.682496:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.682529:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682536:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d6ca lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682557:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682564:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682577:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682582:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302728.682585:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682591:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682599:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302728.682601:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.682604:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682611:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d6d1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682657:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d6c3 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a036c expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682666:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302728.682669:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.682672:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d6c3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a036c expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.682680:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3c00, rc 0) 00000100:00100000:3.0:1789302728.682743:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25863:x1876219350310912:12345-192.168.203.11@tcp:101:bash.0 Request processed in 605us (975us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302728.682759:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 915 00000100:00100000:3.0:1789302728.708374:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.708754:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350311040 00000100:00100000:2.0:1789302728.708763:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350311040 00000100:00100000:2.0:1789302728.708827:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 916 00000100:00100000:2.0:1789302728.708838:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:25864:x1876219350311040:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302728.708861:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302728.708885:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d6d8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0373 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.708966:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.708981:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.709014:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302728.709052:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302728.709056:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302728.709075:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.709100:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d6d8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 38 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.709122:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 37 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.709131:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302728.709134:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302728.709137:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe400, rc 0) 00000100:00100000:2.0:1789302728.709201:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:25864:x1876219350311040:12345-192.168.203.11@tcp:101:rm.0 Request processed in 364us (837us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302728.709208:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 916 00000100:00100000:1.0:1789302728.712849:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302728.712944:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350311168 00000100:00100000:0.0:1789302728.712954:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350311168 00000100:00100000:0.0:1789302728.713015:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 917 00000100:00100000:0.0:1789302728.713026:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+37:25864:x1876219350311168:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302728.713047:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302728.713068:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d6e6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a037a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713131:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713145:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713155:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713170:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713177:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713201:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713299:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d6fb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713308:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d6fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713315:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d6fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302728.713341:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 16384 00000004:00200000:0.0:1789302728.713345:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302728.713362:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d6fb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713377:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713384:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713390:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713396:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302728.713400:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713409:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713430:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:0.0:1789302728.713433:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302728.713439:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713445:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d6ed lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713459:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713465:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713471:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713477:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302728.713480:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713485:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713492:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x12938145 (ffff960f03501b00) 00010000:00010000:0.0:1789302728.713495:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302728.713497:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713544:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d6f4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x12938145 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713596:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d6e6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a037a expref: 39 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713609:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebe00/0x64fe43095184d6fb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a037a expref: 38 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.713617:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:0.0:1789302728.713619:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302728.713622:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebe00, rc 0) 00000100:00100000:0.0:1789302728.713689:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25864:x1876219350311168:12345-192.168.203.11@tcp:101:rm.0 Request processed in 665us (849us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302728.713696:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 917 00000100:00100000:3.0:1789302728.719376:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.719455:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350311296 00000100:00100000:2.0:1789302728.719465:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350311296 00000100:00100000:2.0:1789302728.719572:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 918 00000100:00100000:2.0:1789302728.719582:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25864:x1876219350311296:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302728.719601:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302728.719620:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d702 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0381 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719721:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719735:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719745:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719758:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719765:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719771:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719821:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719828:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719834:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719839:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302728.719843:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719860:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719873:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302728.719876:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302728.719880:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719886:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d709 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719898:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719904:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719909:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719914:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302728.719916:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719921:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.719927:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff960f03927700) 00010000:00010000:2.0:1789302728.720810:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302728.720817:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.720838:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d710 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.720881:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d702 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0381 expref: 39 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.720892:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302728.720894:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302728.720898:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d702 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0381 expref: 39 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.720908:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbec00, rc 0) 00000100:00100000:2.0:1789302728.723264:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25864:x1876219350311296:12345-192.168.203.11@tcp:101:rm.0 Request processed in 3665us (3879us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302728.723277:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 918 00000100:00100000:2.0:1789302728.739055:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302728.739214:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350311424 00000100:00100000:1.0:1789302728.739224:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350311424 00000100:00100000:1.0:1789302728.739287:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 919 00000100:00100000:1.0:1789302728.739298:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25864:x1876219350311424:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302728.739318:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302728.739338:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184d717 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0388 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739402:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739416:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739442:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739458:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739466:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739473:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739587:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739595:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739602:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739608:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302728.739612:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739618:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739638:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302728.739642:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302728.739647:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739660:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d71e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739674:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739680:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739686:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739691:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302728.739695:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739700:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739707:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff960f10d24300) 00010000:00010000:1.0:1789302728.739709:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302728.739712:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739717:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d725 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739751:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184d717 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0388 expref: 39 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739773:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:1.0:1789302728.739776:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302728.739780:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd800/0x64fe43095184d717 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0388 expref: 39 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302728.739789:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dd800, rc 0) 00000100:00100000:1.0:1789302728.739899:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25864:x1876219350311424:12345-192.168.203.11@tcp:101:rm.0 Request processed in 603us (853us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302728.739907:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 919 00000100:00100000:1.0:1789302728.808235:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302728.808411:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350311552 00000100:00100000:3.0:1789302728.808422:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350311552 00000100:00100000:3.0:1789302728.808484:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 920 00000100:00100000:3.0:1789302728.808494:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25865:x1876219350311552:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302728.808567:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302728.808587:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d72c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a038f expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808661:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808676:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808686:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808702:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808709:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808716:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808768:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808790:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808797:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808803:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302728.808808:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808817:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808829:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302728.808832:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.808837:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808845:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d733 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808860:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808867:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808874:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808880:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302728.808889:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808896:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808904:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302728.808906:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.808909:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808914:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d73a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808959:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d72c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a038f expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808968:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302728.808970:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.808974:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d72c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a038f expref: 39 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.808984:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3c00, rc 0) 00000100:00100000:3.0:1789302728.809419:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25865:x1876219350311552:12345-192.168.203.11@tcp:101:cp.0 Request processed in 924us (1191us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302728.809427:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 920 00000100:00100000:1.0:1789302728.833006:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.847053:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350311680 00000100:00100000:2.0:1789302728.847069:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350311680 00000100:00100000:0.0:1789302728.847787:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 921 00000100:00100000:0.0:1789302728.847800:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25865:x1876219350311680:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302728.847822:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302728.847842:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d741 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0396 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.847908:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.847921:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.847945:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.847982:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.847991:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.847999:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848372:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848381:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848387:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848393:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302728.848397:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848404:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848416:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:0.0:1789302728.848419:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302728.848424:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848429:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d748 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848448:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848455:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848461:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848466:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302728.848469:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848476:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848483:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff960f03501b00) 00010000:00010000:0.0:1789302728.848486:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302728.848489:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848495:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d74f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848769:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d741 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0396 expref: 39 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848781:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:0.0:1789302728.848783:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302728.848794:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d741 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0396 expref: 39 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.848804:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302728.849747:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25865:x1876219350311680:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1944us (16746us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302728.849756:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 921 00000100:00100000:0.0:1789302728.874978:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.875176:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350311936 00000100:00100000:2.0:1789302728.875186:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350311936 00000100:00100000:2.0:1789302728.875239:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 922 00000100:00100000:2.0:1789302728.875249:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:25865:x1876219350311936:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302728.875267:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302728.875288:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d756 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a039d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.875421:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.875446:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.875457:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.875472:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.875479:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.875485:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302728.876167:0:5845:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x13:0x0], index 1: 0 00010000:00010000:2.0:1789302728.876361:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d76b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876373:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d76b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876382:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d76b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876414:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876421:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876428:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876440:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302728.876445:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876452:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876464:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302728.876467:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302728.876473:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876479:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184d75d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302728.876492:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ad53a587 reply state 0000000093012472 transno 4294967895 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302728.876525:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302728.876568:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184d756 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a039d expref: 41 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876583:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe800/0x64fe43095184d76b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a039d expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.876591:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:2.0:1789302728.876593:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302728.876595:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe800, rc 0) 00000100:00100000:2.0:1789302728.876657:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25865:x1876219350311936:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1410us (1688us total) trans 4294967895 rc 0/0 00000100:00100000:2.0:1789302728.876671:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 922 00010000:00010000:3.0:1789302728.882201:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.882230:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.882253:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.882259:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302728.882264:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.882272:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.882289:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff960f03927700) 00010000:00010000:3.0:1789302728.882291:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302728.882296:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302728.882302:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184d764 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302728.882871:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.883379:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350312064 00000100:00100000:2.0:1789302728.883390:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350312064 00000100:00100000:2.0:1789302728.883464:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 235 00000100:00100000:2.0:1789302728.883476:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350312064:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302728.883493:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302728.883534:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03a4 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302728.883568:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x13:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302728.883588:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=565s v=(0 0) 00010000:00010000:2.0:1789302728.883593:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302728.883603:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.883628:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.883641:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:2.0:1789302728.883694:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350312064:12345-192.168.203.11@tcp:101:cp.0 Request processed in 221us (827us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302728.883701:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 235 00000100:00080000:3.0:1789302728.883858:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302728.890905:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.891133:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350312192 00000100:00100000:2.0:1789302728.891144:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350312192 00000100:00100000:2.0:1789302728.891211:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 236 00000100:00100000:2.0:1789302728.891221:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:25865:x1876219350312192:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302728.891257:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302728.891277:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03ab expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.891309:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.891318:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302728.891323:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302728.891329:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.891363:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.891379:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 11 pid: 18578 timeout: 665 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302728.891395:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371108608:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302728.891486:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e0281b780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302728.896312:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302728.897961:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371108608:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302728.897999:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302728.898019:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3200, rc 0) 00000100:00100000:2.0:1789302728.898121:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:25865:x1876219350312192:12345-192.168.203.11@tcp:101:cp.0 Request processed in 6901us (7225us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302728.898129:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 236 00000100:00100000:0.0:1789302728.898156:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350312320 00000100:00100000:0.0:1789302728.898164:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350312320 00010000:00010000:0.0:1789302728.898197:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 11 pid: 18578 timeout: 665 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302728.898233:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00100000:0.0:1789302728.898242:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350312320:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302728.898255:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302728.898277:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 11 pid: 18578 timeout: 665 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.898289:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 11 pid: 18578 timeout: 665 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.898313:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184d779 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03a4 expref: 11 pid: 18578 timeout: 665 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.898328:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302728.898339:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302728.898347:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.898356:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302728.898392:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302728.898404:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371108736:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302728.898451:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3300 going to sleep for 11 seconds 00000100:00100000:0.0:1789302728.913706:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371108736:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302728.913734:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302728.913804:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350312320:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 15561us (17508us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302728.913814:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00100000:3.0:1789302728.914126:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302728.914645:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350312576 00000100:00100000:0.0:1789302728.914658:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350312576 00000100:00100000:0.0:1789302728.914727:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 186 00000100:00100000:0.0:1789302728.914739:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:25865:x1876219350312576:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302728.915192:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25865:x1876219350312576:12345-192.168.203.11@tcp:35:cp.0 Request processed in 454us (1073us total) trans 4294967896 rc 0/0 00000100:00100000:0.0:1789302728.915205:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 186 00000100:00100000:3.0:1789302729.010205:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302729.015971:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350312704 00000100:00100000:2.0:1789302729.015988:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350312704 00000100:00100000:2.0:1789302729.016069:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 923 00000100:00100000:2.0:1789302729.016079:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:25868:x1876219350312704:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302729.016099:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302729.016119:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184d787 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03b2 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016187:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016201:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016214:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016230:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016237:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016243:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016326:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d79c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016351:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d79c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016359:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d79c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302729.016395:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 158 00000004:00200000:2.0:1789302729.016399:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302729.016416:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d79c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016428:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d79c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016437:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016444:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016451:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016457:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302729.016462:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016470:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016482:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:2.0:1789302729.016491:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302729.016497:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016537:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184d78e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016564:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016571:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016578:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016583:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302729.016586:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016593:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016601:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x8cab382d (ffff960f03927100) 00010000:00010000:2.0:1789302729.016603:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302729.016607:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.016612:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184d795 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x8cab382d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.019877:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184d787 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03b2 expref: 87 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.019931:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d79c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03b2 expref: 86 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.019940:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:2.0:1789302729.019944:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302729.019947:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3800, rc 0) 00000100:00100000:2.0:1789302729.020092:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25868:x1876219350312704:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4012us (9895us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302729.020102:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 923 00000100:00100000:1.0:1789302729.035563:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302729.039830:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350312832 00000100:00100000:2.0:1789302729.039846:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350312832 00000100:00100000:2.0:1789302729.039941:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 237 00000100:00100000:2.0:1789302729.039954:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350312832:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302729.040091:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302729.040110:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184d7a3 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03b9 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302729.040171:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371108864:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302729.040281:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e0281b780 going to sleep for 11 seconds 00000100:00080000:3.0:1789302729.043907:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1789302729.057084:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302729.057093:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb mtime: 0 -> 1789302728 00002000:00010000:2.0:1789302729.057095:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb atime: 0 -> 1789302728 00002000:00010000:2.0:1789302729.057097:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb ctime: 0 -> 1789302728 00000100:00100000:2.0:1789302729.057119:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371108864:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302729.057154:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184d7a3 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03b9 expref: 13 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.057192:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184d7a3 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03b9 expref: 13 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.057206:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:2.0:1789302729.057269:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350312832:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 17318us (21771us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302729.057278:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 237 00000100:00100000:0.0:1789302729.077596:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302729.078159:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350312960 00000100:00100000:3.0:1789302729.078168:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350312960 00000100:00100000:3.0:1789302729.078234:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 924 00000100:00100000:3.0:1789302729.078245:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25868:x1876219350312960:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302729.078274:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302729.078295:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d7aa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03c0 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.078399:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d7b1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.078413:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d7b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.078426:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d7b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302729.078471:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 16384 00000004:00200000:3.0:1789302729.078476:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302729.078491:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d7b1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.078544:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d7aa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03c0 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.078574:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d7b1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03c0 expref: 87 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.078584:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302729.078586:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302729.078590:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:3.0:1789302729.078664:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:25868:x1876219350312960:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 421us (1078us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.078672:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 924 00000100:00100000:1.0:1789302729.101042:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302729.101468:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350313088 00000100:00100000:2.0:1789302729.101479:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350313088 00000100:00100000:2.0:1789302729.101587:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 238 00000100:00100000:2.0:1789302729.101598:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350313088:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302729.101616:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302729.101646:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184d7b8 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03c7 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302729.101708:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371108992:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302729.102274:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e0281b780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302729.112899:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371108992:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302729.112939:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184d7b8 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03c7 expref: 13 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.112963:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184d7b8 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03c7 expref: 13 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.113003:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:2.0:1789302729.113086:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350313088:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 11488us (12052us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302729.113096:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 238 00000100:00100000:3.0:1789302729.125239:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302729.125382:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350313344 00000100:00100000:0.0:1789302729.125408:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350313344 00000100:00100000:0.0:1789302729.125484:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 925 00000100:00100000:0.0:1789302729.125495:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:25868:x1876219350313344:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:0.0:1789302729.125912:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].23, 136 off11320 00000100:00100000:0.0:1789302729.126005:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:25868:x1876219350313344:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 511us (773us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302729.126013:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 925 00010000:00010000:1.0:1789302729.360759:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302729.550914:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302729.550923:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302729.551086:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 24 count 2 00000040:00080000:0.0:1789302729.551094:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302729.551363:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371109120:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302729.551466:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302729.562280:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302729.562488:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350313472 00000100:00100000:1.0:1789302729.562499:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350313472 00000100:00100000:1.0:1789302729.562577:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 926 00000100:00100000:1.0:1789302729.562589:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25878:x1876219350313472:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302729.562624:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x18:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302729.562738:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25878:x1876219350313472:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 151us (467us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302729.562746:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 926 00000100:00100000:0.0:1789302729.566351:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371109120:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302729.573557:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302729.573765:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350313728 00000100:00100000:0.0:1789302729.573775:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350313728 00000100:00100000:0.0:1789302729.573836:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 927 00000100:00100000:0.0:1789302729.573848:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:25878:x1876219350313728:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302729.573869:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302729.573922:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d7bf lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03ce expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.574022:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d7c6 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.574036:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d7c6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.574058:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d7c6 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302729.574070:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 158 00010000:00010000:0.0:1789302729.574199:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d7bf lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ce expref: 44 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.574218:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d7c6 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ce expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.574226:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:0.0:1789302729.574229:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302729.574233:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:0.0:1789302729.574285:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:25878:x1876219350313728:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 441us (747us total) trans 4294967897 rc 0/0 00000100:00100000:0.0:1789302729.574292:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 927 00000100:00100000:1.0:1789302729.584688:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302729.584739:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302729.585165:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350313984 00000100:00100000:2.0:1789302729.585175:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350313984 00000100:00100000:2.0:1789302729.585246:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 239 00000100:00100000:2.0:1789302729.585257:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350313984:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302729.585301:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.585321:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.585332:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x13:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789302729.585336:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302729.585341:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.585376:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.585391:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 10 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302729.585422:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371109248:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302729.585491:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e0281b780 going to sleep for 11 seconds 00000100:00100000:0.0:1789302729.585668:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350313856 00000100:00100000:0.0:1789302729.585673:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350313856 00000100:00100000:0.0:1789302729.585716:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 38 00000100:00100000:0.0:1789302729.585724:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2348:x1876219350313856:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302729.585734:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302729.585751:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d7c6 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ce expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.585767:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:0.0:1789302729.585772:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302729.585805:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:2348:x1876219350313856:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 81us (1125us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302729.585813:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 38 00000100:00100000:2.0:1789302729.589695:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371109248:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302729.589726:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302729.593579:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302729.593682:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350314112 00000100:00100000:0.0:1789302729.593693:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350314112 00002000:00100000:0.0:1789302729.593743:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x13:0x0] (0->157) req@ffff960f3ef7ce00 x1876219350314112/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:305/0 lens 648/0 e 0 to 0 dl 1789302740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302729.593758:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7ce00 with x1876219350314112 ext(0->157) 00010000:00010000:0.0:1789302729.593768:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 11 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.593785:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 11 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302729.593793:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ef7ce00 00000100:00100000:0.0:1789302729.593829:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 6 00000100:00100000:0.0:1789302729.593839:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350314112:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:0.0:1789302729.594355:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350314112:12345-192.168.203.11@tcp:4:cp.0 Request processed in 516us (784us total) trans 4294967349 rc 0/0 00000100:00100000:0.0:1789302729.594365:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 6 00002000:00100000:0.0:1789302729.594373:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x13:0x0] (0->157) req@ffff960f3ef7ce00 x1876219350314112/t4294967349(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:305/0 lens 648/448 e 0 to 0 dl 1789302740 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302729.594382:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7ce00 with x1876219350314112 ext(0->157) 00010000:00010000:0.0:1789302729.594390:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 13 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.594415:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 13 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302729.594423:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ef7ce00 00000100:00100000:0.0:1789302729.598496:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302729.598615:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350314240 00000100:00100000:3.0:1789302729.598626:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350314240 00010000:00010000:3.0:1789302729.598680:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 13 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302729.598723:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:3.0:1789302729.598734:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350314240:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302729.598750:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302729.598773:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 13 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.598787:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 13 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.598810:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000754ce84f/0x64fe43095184d780 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ab expref: 13 pid: 8571 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.598828:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302729.598832:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302729.598838:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.598848:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.598877:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302729.598932:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b1c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350314240:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 199us (446us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.598940:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 282 00010000:00010000:3.0:1789302729.598994:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599004:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599028:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599036:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599045:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599052:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302729.599057:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599066:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599079:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.599098:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003f9f0da5/0x64fe43095184d7d4 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302729.599144:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350313984:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 13889us (14405us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.599153:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 239 00000100:00100000:1.0:1789302729.616976:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302729.617053:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350314496 00000100:00100000:3.0:1789302729.617064:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350314496 00000100:00100000:3.0:1789302729.617122:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 928 00000100:00100000:3.0:1789302729.617132:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25878:x1876219350314496:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302729.617245:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25878:x1876219350314496:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 113us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.617253:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 928 00000100:00100000:3.0:1789302729.626078:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302729.626247:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350314624 00000100:00100000:1.0:1789302729.626268:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350314624 00000100:00100000:0.0:1789302729.627388:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 240 00000100:00100000:0.0:1789302729.627409:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350314624:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302729.627428:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302729.627452:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03d5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302729.627481:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x13:0x0] initial LVB size: 158, mtime: 0x6aa697c8, atime: 0x0, ctime: 0x6aa697c8, blocks: 0x1 00010000:00010000:0.0:1789302729.627538:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=566s v=(0 0) 00010000:00010000:0.0:1789302729.627554:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302729.627564:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.627581:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302729.627593:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:0.0:1789302729.627664:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350314624:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 263us (1597us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302729.627671:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 240 00000100:00100000:1.0:1789302729.639039:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302729.639109:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350314880 00000100:00100000:3.0:1789302729.639119:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350314880 00000100:00100000:3.0:1789302729.639171:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 929 00000100:00100000:3.0:1789302729.639181:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25878:x1876219350314880:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302729.639308:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25878:x1876219350314880:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 128us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.639315:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 929 00000100:00100000:1.0:1789302729.647110:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302729.647235:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350315008 00000100:00100000:0.0:1789302729.647249:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350315008 00002000:00100000:0.0:1789302729.647301:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x13:0x0] (0->4095) req@ffff960f3ad45880 x1876219350315008/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:305/0 lens 488/0 e 0 to 0 dl 1789302740 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302729.647312:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad45880 with x1876219350315008 ext(0->4095) 00002000:00010000:0.0:1789302729.647318:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ad45880 00000100:00100000:0.0:1789302729.647355:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 89 00000100:00100000:0.0:1789302729.647365:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350315008:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302729.647609:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350315008:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 245us (511us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302729.648309:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 89 00002000:00100000:0.0:1789302729.648325:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x13:0x0] (0->4095) req@ffff960f3ad45880 x1876219350315008/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:305/0 lens 488/4536 e 0 to 0 dl 1789302740 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302729.648336:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad45880 with x1876219350315008 ext(0->4095) 00002000:00010000:0.0:1789302729.648341:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ad45880 00000100:00100000:2.0:1789302729.654811:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302729.654918:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350315136 00000100:00100000:3.0:1789302729.654929:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350315136 00000100:00100000:3.0:1789302729.654986:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 930 00000100:00100000:3.0:1789302729.655010:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:25878:x1876219350315136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302729.655029:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302729.655049:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d7e2 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03dc expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.655080:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.655093:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.655103:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.655191:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095184d7e2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 44 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.655369:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 43 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.655377:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:3.0:1789302729.655380:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302729.655383:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2800, rc 0) 00000100:00100000:3.0:1789302729.655478:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:25878:x1876219350315136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 469us (675us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.655485:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 930 00000100:00100000:1.0:1789302729.660944:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302729.661042:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350315264 00000100:00100000:2.0:1789302729.661053:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350315264 00000100:00100000:2.0:1789302729.661112:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 931 00000100:00100000:2.0:1789302729.661122:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:25878:x1876219350315264:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302729.661155:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x18:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302729.661266:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:25878:x1876219350315264:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 139us (324us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302729.661272:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 931 00000100:00100000:1.0:1789302729.665182:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302729.665365:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350315392 00000100:00100000:2.0:1789302729.665374:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350315392 00000100:00100000:2.0:1789302729.665457:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 241 00000100:00100000:2.0:1789302729.665467:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350315392:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302729.665554:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665572:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302729.665591:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302729.665597:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665612:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665746:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665756:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665764:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665771:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302729.665776:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665784:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665799:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.665806:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000087c404b2/0x64fe43095184d7f0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302729.665916:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350315392:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 450us (743us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302729.665924:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 241 00000100:00100000:0.0:1789302729.672779:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302729.673789:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350315648 00000100:00100000:2.0:1789302729.673803:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350315648 00000100:00100000:2.0:1789302729.673925:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 932 00000100:00100000:2.0:1789302729.673938:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:25878:x1876219350315648:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302729.674029:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.674047:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 43 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.674056:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.674090:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 43 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.674106:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 43 pid: 6919 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302729.674120:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371109376:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302729.674224:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e0281b3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302729.684161:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371109376:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302729.684215:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302729.700099:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302729.700323:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350315776 00000100:00100000:3.0:1789302729.700333:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350315776 00010000:00010000:3.0:1789302729.700373:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 44 pid: 6919 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302729.700433:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:3.0:1789302729.700442:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350315776:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302729.700458:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302729.700465:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 44 pid: 6919 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.700477:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 44 pid: 6919 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.700553:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095184d7e9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03dc expref: 44 pid: 6919 timeout: 666 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.700570:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:3.0:1789302729.700576:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.700585:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.700593:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302729.700622:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302729.700677:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2347:x1876219350315776:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 236us (588us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.700684:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 283 00010000:00010000:2.0:1789302729.701078:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701087:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701289:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701298:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701305:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701311:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302729.701316:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701325:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701335:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:2.0:1789302729.701338:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302729.701342:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302729.701348:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184d7f7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x18:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302729.701486:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:25878:x1876219350315648:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 27551us (28716us total) trans 4294967898 rc 0/0 00000100:00100000:2.0:1789302729.701526:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 932 00000040:00080000:1.0:1789302729.705386:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302729.705396:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302729.705934:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:1.0:1789302729.705944:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302729.722600:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302729.723552:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350316032 00000100:00100000:0.0:1789302729.723563:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350316032 00000100:00100000:3.0:1789302729.723882:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 187 00000100:00100000:3.0:1789302729.723894:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:25878:x1876219350316032:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302729.724139:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:25878:x1876219350316032:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 246us (1546us total) trans 4294967899 rc 0/0 00000100:00100000:3.0:1789302729.724147:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 187 00000100:00100000:3.0:1789302729.745948:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371109504:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746030:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302729.746124:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371109888:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746157:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302729.746220:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371110016:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746248:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302729.746285:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371110016 00000100:00100000:3.0:1789302729.746294:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371110016 00000100:00100000:3.0:1789302729.746351:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 363 00000100:00100000:3.0:1789302729.746362:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371110016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746411:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371110016:12345-0@lo:400:kworker.0 Request processed in 51us (163us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.746418:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 363 00000100:00080000:3.0:1789302729.746436:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302729.746462:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371110016:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746566:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371109632:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746589:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302729.746604:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371110144:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746762:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302729.746796:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302729.746803:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371109760:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746824:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302729.746850:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371109504 00000100:00100000:3.0:1789302729.746854:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371109504 00000100:00100000:3.0:1789302729.746879:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:3.0:1789302729.746885:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371109504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.746915:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371109504:12345-0@lo:400:kworker.0 Request processed in 31us (886us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.746920:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:3.0:1789302729.746944:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371109888 00000100:00100000:3.0:1789302729.746946:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371109888 00000100:00100000:3.0:1789302729.746970:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371110144 00000100:00100000:3.0:1789302729.746972:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371110144 00000100:00100000:3.0:1789302729.746989:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1789302729.746993:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371109888:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747021:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371109888:12345-0@lo:400:kworker.0 Request processed in 27us (863us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.747027:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1789302729.747036:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 349 00000100:00100000:3.0:1789302729.747041:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371110144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747064:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371110144:12345-0@lo:400:kworker.0 Request processed in 23us (302us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.747069:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 349 00000100:00080000:3.0:1789302729.747081:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302729.747084:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302729.747102:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371109888:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747136:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371109632 00000100:00100000:3.0:1789302729.747140:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371109632 00000100:00100000:3.0:1789302729.747160:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371109760 00000100:00100000:3.0:1789302729.747163:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371109760 00000100:00100000:3.0:1789302729.747182:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:3.0:1789302729.747188:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371109632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747441:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371109632:12345-0@lo:400:kworker.0 Request processed in 251us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.747447:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:3.0:1789302729.747460:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:3.0:1789302729.747466:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371109760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747488:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371109760:12345-0@lo:400:kworker.0 Request processed in 23us (663us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302729.747493:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00080000:3.0:1789302729.747535:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302729.747565:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371109760:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747575:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371109632:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747584:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371110144:0@lo:400:kworker.0 00000100:00100000:3.0:1789302729.747607:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371109504:0@lo:400:kworker.0 00000100:00100000:0.0:1789302729.997788:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302729.997826:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350316160 00000100:00100000:0.0:1789302729.997835:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350316160 00000100:00100000:0.0:1789302729.997905:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 547 00000100:00100000:0.0:1789302729.997914:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350316160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302729.997981:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350316160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302729.998007:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 547 00000100:00100000:1.0:1789302730.018089:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302730.018158:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350316288 00000100:00100000:3.0:1789302730.018168:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350316288 00000100:00100000:3.0:1789302730.018252:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:3.0:1789302730.018263:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:2348:x1876219350316288:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302730.018335:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:2348:x1876219350316288:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 74us (256us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302730.018350:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00080000:0.0:1789302730.018610:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302730.031952:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302730.031998:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302730.032077:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350316544 00000100:00100000:1.0:1789302730.032087:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350316544 00000100:00100000:1.0:1789302730.032140:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350316800 00000100:00100000:1.0:1789302730.032144:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350316800 00000100:00100000:1.0:1789302730.032181:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 350 00000100:00100000:1.0:1789302730.032191:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350316544:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302730.032273:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350316544:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 84us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302730.032281:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 350 00000100:00100000:1.0:1789302730.032296:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 351 00000100:00100000:1.0:1789302730.032302:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350316800:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302730.032327:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350316800:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (329us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302730.032353:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 351 00000100:00080000:1.0:1789302730.032374:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302730.058208:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302730.058248:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302730.058319:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350316416 00000100:00100000:1.0:1789302730.058330:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350316416 00000100:00100000:1.0:1789302730.058392:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350316672 00000100:00100000:1.0:1789302730.058395:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350316672 00000100:00100000:1.0:1789302730.058419:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 352 00000100:00100000:1.0:1789302730.058429:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350316416:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302730.058498:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350316416:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 71us (312us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302730.058527:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 352 00000100:00100000:1.0:1789302730.058553:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 353 00000100:00100000:1.0:1789302730.058561:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350316672:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302730.058585:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350316672:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (337us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302730.058591:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 353 00000100:00080000:0.0:1789302730.058955:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302730.257101:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789302730.258272:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371110272:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302730.258355:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302730.258455:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371110272 00000100:00100000:0.0:1789302730.258462:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371110272 00000100:00100000:0.0:1789302730.260852:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 156 00000100:00100000:0.0:1789302730.260873:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371110272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302730.260901:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 73072640 pending 1703936 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302730.260978:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371110272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 114us (2622us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302730.260986:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 156 00000100:00100000:0.0:1789302730.261045:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371110272:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789302730.375696:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967349 is committed 00010000:00010000:0.0:1789302730.381301:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302730.388189:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967897 is committed 00000001:00080000:2.0:1789302730.388221:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967898 is committed 00000001:00080000:2.0:1789302730.388227:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967899 is committed 00000001:00080000:2.0:1789302730.388232:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967894 is committed 00000040:00080000:1.0:1789302730.439475:0:19021:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302730.439485:0:19021:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302730.439670:0:19021:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 24 count 2 00000040:00080000:1.0:1789302730.439676:0:19021:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302730.513682:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302730.513783:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350317056 00000100:00100000:3.0:1789302730.513795:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350317056 00000100:00100000:3.0:1789302730.513859:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 933 00000100:00100000:3.0:1789302730.513869:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25879:x1876219350317056:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302730.513892:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302730.513896:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:3.0:1789302730.513922:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095184d7b1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03c0 expref: 86 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.513944:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:3.0:1789302730.513949:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302730.513963:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d7fe lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03e3 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.514083:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d805 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.514097:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d805 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.514108:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d805 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302730.514119:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 158 00010000:00010000:3.0:1789302730.514330:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d7fe lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03e3 expref: 88 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.514344:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d805 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03e3 expref: 87 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.514351:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:3.0:1789302730.514354:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302730.514357:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302730.514411:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:25879:x1876219350317056:12345-192.168.203.11@tcp:101:bash.0 Request processed in 544us (740us total) trans 4294967900 rc 0/0 00000100:00100000:3.0:1789302730.514418:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 933 00000100:00100000:2.0:1789302730.528310:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302730.528462:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350317184 00000100:00100000:3.0:1789302730.528472:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350317184 00000100:00100000:3.0:1789302730.528559:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 39 00000100:00100000:3.0:1789302730.528569:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:2347:x1876219350317184:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302730.528586:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302730.528610:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d805 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03e3 expref: 87 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.528638:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:3.0:1789302730.528644:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302730.528699:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:2347:x1876219350317184:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 132us (398us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302730.528707:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 39 00000100:00100000:0.0:1789302730.559051:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302730.559356:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350317312 00000100:00100000:2.0:1789302730.559366:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350317312 00000100:00100000:2.0:1789302730.559428:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:2.0:1789302730.559439:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350317312:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302730.559458:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302730.559477:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03ea expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.559549:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789302730.559554:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302730.559563:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 13 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.559581:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 13 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.559594:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302730.559669:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350317312:12345-192.168.203.11@tcp:101:cat.0 Request processed in 232us (628us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302730.559677:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:3.0:1789302730.568610:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302730.568693:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350317440 00000100:00100000:1.0:1789302730.568704:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350317440 00000100:00100000:1.0:1789302730.568768:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:1.0:1789302730.568779:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:25879:x1876219350317440:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302730.568797:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302730.568828:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a0b20dc8/0x64fe43095184d81a lrc: 2/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03f1 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.568860:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.568870:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 14 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.568877:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302730.568881:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302730.568887:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a0b20dc8/0x64fe43095184d81a lrc: 5/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f1 expref: 14 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.568920:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 14 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.568932:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 14 pid: 18578 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302730.568946:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371110400:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302730.569044:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 9 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.569054:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 9 pid: 6697 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302730.569077:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371110528:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302730.569103:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302730.576259:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371110400:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302730.576913:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371110528:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302730.576946:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a0b20dc8/0x64fe43095184d81a lrc: 3/0,0 mode: --/PW res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f1 expref: 14 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.576967:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dd800, rc 0) 00000100:00100000:1.0:1789302730.577022:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:25879:x1876219350317440:12345-192.168.203.11@tcp:101:cat.0 Request processed in 8244us (8420us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302730.577030:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:0.0:1789302730.584669:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302730.584714:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302730.584756:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350317568 00000100:00100000:3.0:1789302730.584774:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350317568 00000100:00100000:2.0:1789302730.584784:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350317696 00000100:00100000:2.0:1789302730.584788:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350317696 00010000:00010000:2.0:1789302730.584817:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 10 pid: 6697 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302730.584862:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:2.0:1789302730.584872:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350317696:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302730.584887:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:2.0:1789302730.584907:0:5830:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x13:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:2.0:1789302730.584915:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 10 pid: 6697 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.584928:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 10 pid: 6697 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.584951:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b90e3dee/0x64fe43095184d7db lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03d5 expref: 10 pid: 6697 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.584968:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302730.585007:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 14 pid: 18578 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302730.585022:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350317696:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 152us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302730.585029:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:3.0:1789302730.585482:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 285 00000100:00100000:3.0:1789302730.585493:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350317568:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302730.585528:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302730.585552:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 14 pid: 18578 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.585564:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 14 pid: 18578 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.585577:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184d813 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x13:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ea expref: 14 pid: 18578 timeout: 667 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.585588:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302730.585592:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x13:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302730.585599:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a0b20dc8/0x64fe43095184d81a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f1 expref: 13 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.585609:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a0b20dc8/0x64fe43095184d81a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f1 expref: 13 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.586829:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a0b20dc8/0x64fe43095184d81a lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x13:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f1 expref: 13 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302730.586863:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371110656:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302730.586951:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33589d80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302730.594957:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302730.595026:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350317824 00000100:00100000:1.0:1789302730.595034:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350317824 00000100:00100000:3.0:1789302730.595040:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371110656:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302730.595061:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00002000:00100000:1.0:1789302730.595078:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (0->4095) req@ffff960f3f7c9500 x1876219350317824/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:306/0 lens 488/0 e 0 to 0 dl 1789302741 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302730.595087:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7c9500 with x1876219350317824 ext(0->4095) 00002000:00010000:1.0:1789302730.595092:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f7c9500 00000100:00100000:3.0:1789302730.595117:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350317568:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 9618us (10449us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302730.595126:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 90 00000100:00100000:3.0:1789302730.595127:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 285 00000100:00100000:1.0:1789302730.595135:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2347:x1876219350317824:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:1.0:1789302730.595353:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350317824:12345-192.168.203.11@tcp:3:cat.0 Request processed in 219us (405us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302730.595361:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 90 00002000:00100000:1.0:1789302730.595376:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x13:0x0] (0->4095) req@ffff960f3f7c9500 x1876219350317824/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:306/0 lens 488/4536 e 0 to 0 dl 1789302741 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302730.595384:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7c9500 with x1876219350317824 ext(0->4095) 00002000:00010000:1.0:1789302730.595387:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f7c9500 00000100:00100000:0.0:1789302730.602823:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302730.602979:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350318080 00000100:00100000:2.0:1789302730.603005:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350318080 00000100:00100000:2.0:1789302730.603061:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 188 00000100:00100000:2.0:1789302730.603072:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:25879:x1876219350318080:12345-192.168.203.11@tcp:35:cat.0 00000001:00080000:2.0:1789302730.603363:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967901 00000001:00080000:2.0:1789302730.603423:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967902 00000100:00100000:2.0:1789302730.603919:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25879:x1876219350318080:12345-192.168.203.11@tcp:35:cat.0 Request processed in 846us (1102us total) trans 4294967903 rc 0/0 00000100:00100000:2.0:1789302730.603928:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 188 00000020:00080000:2.0:1789302730.603934:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 3 transactions per RPC 00000100:00100000:3.0:1789302730.662094:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302730.662224:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350318208 00000100:00100000:0.0:1789302730.662234:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350318208 00002000:00100000:0.0:1789302730.662282:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (0->315) req@ffff960f1db44700 x1876219350318208/t0(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:306/0 lens 808/0 e 0 to 0 dl 1789302741 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302730.662312:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1db44700 with x1876219350318208 ext(0->315) 00002000:00010000:0.0:1789302730.662318:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1db44700 00000100:00100000:0.0:1789302730.662354:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 91 00000100:00100000:0.0:1789302730.662364:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2346:x1876219350318208:12345-192.168.203.11@tcp:4:cat.0 00000100:00100000:0.0:1789302730.662873:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2346:x1876219350318208:12345-192.168.203.11@tcp:4:cat.0 Request processed in 510us (785us total) trans 4294967350 rc 0/0 00000100:00100000:0.0:1789302730.662881:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 91 00002000:00100000:0.0:1789302730.662888:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x13:0x0] (0->315) req@ffff960f1db44700 x1876219350318208/t4294967350(0) o4->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:306/0 lens 808/448 e 0 to 0 dl 1789302741 ref 1 fl Complete:/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302730.662897:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1db44700 with x1876219350318208 ext(0->315) 00002000:00010000:0.0:1789302730.662900:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1db44700 00000100:00100000:3.0:1789302730.676867:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302730.677305:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350318464 00000100:00100000:1.0:1789302730.677318:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350318464 00000100:00100000:1.0:1789302730.677383:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 934 00000100:00100000:1.0:1789302730.677394:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:25881:x1876219350318464:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302730.677420:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302730.677440:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d821 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03f8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.677586:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d828 lrc: 3/1,0 mode: --/CR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.677601:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d828 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.677611:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d828 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302730.677622:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x18:0x0]: returning size 316 00010000:00010000:1.0:1789302730.677743:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d821 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3b:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f8 expref: 92 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.677759:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d828 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f8 expref: 91 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302730.677768:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:1.0:1789302730.677782:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:1.0:1789302730.677786:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120ddc00, rc 0) 00000100:00100000:1.0:1789302730.677850:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:25881:x1876219350318464:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 458us (992us total) trans 4294967904 rc 0/0 00000100:00100000:1.0:1789302730.677857:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 934 00000100:00100000:1.0:1789302730.685755:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302730.685860:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350318592 00000100:00100000:3.0:1789302730.685871:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350318592 00000100:00100000:3.0:1789302730.685934:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 935 00000100:00100000:3.0:1789302730.685943:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:25881:x1876219350318592:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:3.0:1789302730.686009:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686026:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686037:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686063:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686069:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686091:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686096:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302730.686100:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686106:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686118:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:3.0:1789302730.686121:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302730.686126:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302730.686133:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d836 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302730.686224:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:25881:x1876219350318592:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 282us (480us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302730.686232:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 935 00000100:00100000:2.0:1789302730.716839:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302730.717214:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350318720 00000100:00100000:0.0:1789302730.717223:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350318720 00000100:00100000:0.0:1789302730.717377:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 936 00000100:00100000:0.0:1789302730.717386:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:25882:x1876219350318720:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302730.717401:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302730.717418:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d83d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a03ff expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302730.717468:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d844 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302730.717478:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d844 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302730.717497:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d844 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302730.717554:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3b:0x0]: returning size 16384 00000004:00200000:0.0:1789302730.717561:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302730.717579:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d844 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302730.717604:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d83d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ff expref: 93 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302730.717620:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb000/0x64fe43095184d844 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03ff expref: 92 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302730.717628:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3b:0x0].0x0 (ffff960f10d24f00) 00010000:00010000:0.0:1789302730.717631:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302730.717634:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb000, rc 0) 00000100:00100000:0.0:1789302730.717699:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:25882:x1876219350318720:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 313us (870us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302730.717707:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 936 00000100:00100000:1.0:1789302730.734266:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302730.734337:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350319104 00000100:00100000:0.0:1789302730.734349:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350319104 00000100:00100000:0.0:1789302730.734414:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 189 00000100:00100000:0.0:1789302730.734425:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:25882:x1876219350319104:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:0.0:1789302730.734699:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:25882:x1876219350319104:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 275us (443us total) trans 4294967905 rc 0/0 00000100:00100000:0.0:1789302730.734707:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 189 00000100:00100000:0.0:1789302730.737849:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302730.737931:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350319232 00000100:00100000:2.0:1789302730.737941:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350319232 00000100:00100000:2.0:1789302730.738008:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 40 00000100:00100000:2.0:1789302730.738018:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:25882:x1876219350319232:12345-192.168.203.11@tcp:103:lfs.0 00010000:00010000:2.0:1789302730.738033:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1789302730.738058:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184d828 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x18:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03f8 expref: 93 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.738090:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d79c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x18:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a03b2 expref: 92 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302730.738100:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x18:0x0].0x0 (ffff960f03927900) 00010000:00010000:2.0:1789302730.738104:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302730.738156:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:25882:x1876219350319232:12345-192.168.203.11@tcp:103:lfs.0 Request processed in 140us (317us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302730.738163:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 40 00000100:00100000:2.0:1789302730.754367:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302730.755046:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350319360 00000100:00100000:2.0:1789302730.755059:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350319360 00000100:00100000:2.0:1789302730.755116:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 937 00000100:00100000:2.0:1789302730.755126:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:25882:x1876219350319360:12345-192.168.203.11@tcp:58:lfs.0 00000100:00100000:2.0:1789302730.755680:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:25882:x1876219350319360:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 553us (1319us total) trans 0 rc -1/-1 00000100:00100000:2.0:1789302730.755688:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 937 00000100:00100000:3.0:1789302730.767565:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371110784:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302730.767653:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302730.768381:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371110784 00000100:00100000:0.0:1789302730.768389:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371110784 00000100:00100000:0.0:1789302730.768440:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:0.0:1789302730.768449:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371110784:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302730.768486:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 73072640 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1789302730.768573:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371110784:12345-0@lo:13:osp-pre-0-0.0 Request processed in 124us (919us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302730.768581:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1789302730.769327:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371110784:0@lo:13:osp-pre-0-0.0 00010000:00010000:0.0:1789302731.404876:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302732.428855:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302733.470650:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302734.477452:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302734.862210:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371110912:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862280:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302734.862331:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371111424:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862386:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302734.862413:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302734.862419:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371111296:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862441:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302734.862469:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371111040:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862492:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302734.862538:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371111552:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862557:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302734.862573:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302734.862578:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371111168:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862597:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302734.862624:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371110912 00000100:00100000:3.0:1789302734.862631:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371110912 00000100:00100000:3.0:1789302734.862676:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 548 00000100:00100000:3.0:1789302734.862684:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371110912:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862732:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371110912:12345-0@lo:400:kworker.0 Request processed in 49us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302734.862739:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 548 00000100:00100000:3.0:1789302734.862769:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371111424 00000100:00100000:3.0:1789302734.862772:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371111424 00000100:00100000:3.0:1789302734.862803:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371111040 00000100:00100000:3.0:1789302734.862805:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371111040 00000100:00100000:3.0:1789302734.862818:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371111168 00000100:00100000:3.0:1789302734.862820:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371111168 00000100:00100000:3.0:1789302734.862834:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 367 00000100:00100000:3.0:1789302734.862840:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371111424:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862870:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371111424:12345-0@lo:400:kworker.0 Request processed in 30us (483us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302734.862882:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 367 00000100:00100000:3.0:1789302734.862896:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1789302734.862901:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371111040:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862932:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371111040:12345-0@lo:400:kworker.0 Request processed in 30us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302734.862937:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 368 00000100:00100000:3.0:1789302734.862948:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 369 00000100:00100000:3.0:1789302734.862953:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371111168:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.862977:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371111168:12345-0@lo:400:kworker.0 Request processed in 24us (379us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302734.862983:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 369 00000100:00100000:3.0:1789302734.862999:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371111296 00000100:00100000:3.0:1789302734.863003:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371111296 00000100:00100000:3.0:1789302734.863026:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371111552 00000100:00100000:3.0:1789302734.863030:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371111552 00000100:00100000:3.0:1789302734.863045:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1789302734.863051:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371111296:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.863080:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371111296:12345-0@lo:400:kworker.0 Request processed in 30us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302734.863088:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1789302734.863103:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 355 00000100:00100000:3.0:1789302734.863110:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371111552:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.863134:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371111552:12345-0@lo:400:kworker.0 Request processed in 25us (575us total) trans 0 rc 0/0 00000100:00080000:1.0:1789302734.863138:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302734.863142:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 355 00000100:00080000:1.0:1789302734.863145:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302734.863148:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302734.863192:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371111296:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.863202:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371110912:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.863211:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371111424:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.863240:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371111168:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.863248:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371111040:0@lo:400:kworker.0 00000100:00100000:3.0:1789302734.863256:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371111552:0@lo:400:kworker.0 00000100:00100000:2.0:1789302735.137562:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302735.137618:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302735.137654:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302735.137692:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302735.137711:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302735.137732:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302735.137755:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350319744 00000100:00100000:2.0:1789302735.137756:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302735.137765:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350319744 00000100:00100000:2.0:1789302735.137775:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350320256 00000100:00100000:2.0:1789302735.137778:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350320256 00000100:00100000:3.0:1789302735.137827:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350319872 00000100:00100000:3.0:1789302735.137831:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350319872 00000100:00100000:0.0:1789302735.137835:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350319488 00000100:00100000:0.0:1789302735.137839:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350319488 00000100:00100000:1.0:1789302735.137886:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350320000 00000100:00100000:1.0:1789302735.137890:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350320000 00000100:00100000:0.0:1789302735.137938:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 549 00000100:00100000:0.0:1789302735.137947:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350319488:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302735.138012:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350319488:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 66us (459us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302735.138019:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 549 00000100:00100000:0.0:1789302735.138072:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350320128 00000100:00100000:0.0:1789302735.138075:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350320128 00000100:00100000:2.0:1789302735.138704:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 356 00000100:00100000:2.0:1789302735.138714:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350319744:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302735.138751:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350319744:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 40us (1133us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302735.138757:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 356 00000100:00100000:2.0:1789302735.138769:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:2.0:1789302735.138774:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350320256:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302735.138797:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350320256:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (1105us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302735.138820:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:2.0:1789302735.138959:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350319616 00000100:00100000:2.0:1789302735.138965:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350319616 00000100:00100000:2.0:1789302735.138997:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 370 00000100:00100000:2.0:1789302735.139005:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2345:x1876219350319616:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302735.139039:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2345:x1876219350319616:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (1327us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302735.139045:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 370 00000100:00080000:2.0:1789302735.139101:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302735.139106:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302735.139108:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302735.140954:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 358 00000100:00100000:3.0:1789302735.140967:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2345:x1876219350319872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302735.141026:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2345:x1876219350319872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 63us (3292us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302735.141033:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 358 00000100:00100000:1.0:1789302735.142674:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 371 00000100:00100000:1.0:1789302735.142689:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2345:x1876219350320000:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302735.142763:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2345:x1876219350320000:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 77us (5106us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302735.142770:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 371 00000100:00100000:0.0:1789302735.142876:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 359 00000100:00100000:0.0:1789302735.142885:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350320128:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302735.142921:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350320128:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (5164us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302735.142927:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 359 00000100:00100000:3.0:1789302735.376002:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302735.376023:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371111680:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302735.376089:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302735.376153:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371111680 00000100:00100000:3.0:1789302735.376172:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371111680 00000100:00100000:3.0:1789302735.376215:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 158 00000100:00100000:3.0:1789302735.376224:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371111680:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302735.376253:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 74776576 pending 1703936 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302735.376292:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371111680:12345-0@lo:13:osp-pre-1-0.0 Request processed in 69us (203us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302735.376306:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 158 00000100:00100000:3.0:1789302735.376357:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371111680:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789302735.460839:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967350 is committed 00000001:00080000:3.0:1789302735.490354:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967905 is committed 00010000:00010000:1.0:1789302735.501049:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302735.884832:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371111808:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302735.884899:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302735.885476:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371111808 00000100:00100000:0.0:1789302735.885486:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371111808 00000100:00100000:0.0:1789302735.885571:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:0.0:1789302735.885592:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371111808:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302735.885625:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 73072640 pending 0 free 3857711104 avail 3855613952 00000100:00100000:0.0:1789302735.885684:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371111808:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (784us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302735.885690:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:3.0:1789302735.885889:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371111808:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302736.525699:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302736.788356:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302736.788552:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350320384 00000100:00100000:2.0:1789302736.788562:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350320384 00000100:00100000:2.0:1789302736.788623:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 938 00000100:00100000:2.0:1789302736.788633:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:25861:x1876219350320384:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1789302736.788735:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:25861:x1876219350320384:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 103us (387us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302736.788742:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 938 00010000:00010000:2.0:1789302737.549166:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302738.572995:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302739.157698:0:19221:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12f: Restore a released file explicitly ========================================================== 08:32:16 (1789302736) 00000100:00100000:0.0:1789302739.575681:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.575950:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350320512 00000100:00100000:2.0:1789302739.575963:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350320512 00000100:00100000:2.0:1789302739.576061:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 939 00000100:00100000:2.0:1789302739.576074:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:26485:x1876219350320512:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302739.576259:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:26485:x1876219350320512:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 188us (593us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.576268:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 939 00000100:00080000:0.0:1789302739.578702:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302739.597754:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302739.608423:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302739.608560:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350320640 00000100:00100000:1.0:1789302739.608574:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350320640 00000100:00100000:1.0:1789302739.608664:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 940 00000100:00100000:1.0:1789302739.608676:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:26486:x1876219350320640:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302739.608705:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302739.608750:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d84b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0406 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789302739.608938:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302739.608970:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.608991:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609006:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609159:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609182:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609194:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609678:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609688:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609694:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609700:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302739.609704:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609722:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609734:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:1.0:1789302739.609737:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302739.609742:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609747:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d852 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609760:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609765:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609770:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609774:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302739.609777:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609782:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609788:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff960f10d24000) 00010000:00010000:1.0:1789302739.609790:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302739.609793:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609807:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d859 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609960:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d84b lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0406 expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609971:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:1.0:1789302739.609973:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302739.609977:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d84b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0406 expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.609988:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f800, rc 0) 00000100:00100000:1.0:1789302739.610116:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:26486:x1876219350320640:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1441us (1708us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302739.610126:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 940 00000100:00100000:1.0:1789302739.614727:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.614949:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350320896 00000100:00100000:2.0:1789302739.614966:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350320896 00000100:00100000:2.0:1789302739.615055:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 941 00000100:00100000:2.0:1789302739.615071:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:26486:x1876219350320896:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:2.0:1789302739.615146:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302739.615181:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184d6a0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a035e expref: 85 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.615204:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302739.615210:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302739.617148:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.617552:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 38 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.617563:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.617616:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 38 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.617636:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 38 pid: 18427 timeout: 676 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302739.617667:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371111936:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302739.617812:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302739.623088:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.623595:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371111936:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302739.623641:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302739.627983:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321024 00000100:00100000:3.0:1789302739.628007:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321024 00010000:00010000:3.0:1789302739.628082:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 39 pid: 18427 timeout: 676 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302739.628727:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 286 00000100:00100000:3.0:1789302739.630021:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2347:x1876219350321024:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302739.630047:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302739.630060:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 39 pid: 18427 timeout: 676 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.630081:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 39 pid: 18427 timeout: 676 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.630110:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184d6df lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0373 expref: 39 pid: 18427 timeout: 676 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.630129:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:3.0:1789302739.630136:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.630162:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.630171:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.630212:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302739.630291:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:2347:x1876219350321024:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 290us (7218us total) trans 0 rc 0/0 00010000:00010000:2.0:1789302739.635569:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.635593:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.635638:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.635655:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.635670:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302739.637043:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 16384 00010000:00010000:2.0:1789302739.637061:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.637071:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.637079:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.637094:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302739.637100:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.637108:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.637127:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302739.637131:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.637136:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.637144:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d860 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302739.637167:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000073578821 reply state 00000000414b50bc transno 4294967906 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302739.637178:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302739.637310:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:26486:x1876219350320896:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 22239us (22606us total) trans 4294967906 rc 0/0 00000100:00100000:2.0:1789302739.637321:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 941 00010000:00010000:0.0:1789302739.642498:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.642559:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.642569:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.642576:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302739.642581:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.642591:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.642611:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff960f03927f00) 00010000:00010000:0.0:1789302739.642615:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.642621:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.642628:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184d867 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302739.649036:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 286 00000100:00100000:1.0:1789302739.652128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.654750:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321152 00000100:00100000:3.0:1789302739.654770:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321152 00000100:00100000:3.0:1789302739.654861:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 942 00000100:00100000:3.0:1789302739.654874:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+85:26487:x1876219350321152:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302739.654908:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302739.654939:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d86e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a040d expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.655037:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d875 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.655056:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d875 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.655070:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d875 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302739.655122:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302739.655454:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302739.655553:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d875 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.655585:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d86e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a040d expref: 87 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.655614:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d875 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a040d expref: 86 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.655624:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302739.655628:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302739.655633:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0a00, rc 0) 00000100:00100000:3.0:1789302739.655709:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:26487:x1876219350321152:12345-192.168.203.11@tcp:101:bash.0 Request processed in 838us (3597us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302739.655718:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 942 00000100:00100000:1.0:1789302739.660798:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.661003:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321280 00000100:00100000:2.0:1789302739.661020:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321280 00000100:00100000:2.0:1789302739.661092:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 943 00000100:00100000:2.0:1789302739.661104:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:26487:x1876219350321280:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302739.661142:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302739.661167:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d87c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0414 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.661273:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.661291:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.661305:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.661321:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.661329:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.661335:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662295:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2e00/0x64fe43095184d891 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662307:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2e00/0x64fe43095184d891 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662317:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2e00/0x64fe43095184d891 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302739.662356:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 16384 00000004:00200000:2.0:1789302739.662360:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302739.662384:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2e00/0x64fe43095184d891 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662415:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662422:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662934:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662940:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302739.662945:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662953:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662968:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302739.662971:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.662977:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662983:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d883 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.662998:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663005:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663010:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663026:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302739.663029:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663035:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663042:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff960f03927400) 00010000:00010000:2.0:1789302739.663045:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.663047:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663053:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d88a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663102:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d87c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0414 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663118:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2e00/0x64fe43095184d891 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0414 expref: 87 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.663125:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302739.663964:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302739.663969:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2e00, rc 0) 00000100:00100000:2.0:1789302739.664116:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:26487:x1876219350321280:12345-192.168.203.11@tcp:101:bash.0 Request processed in 3010us (3330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.664136:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 943 00000100:00100000:1.0:1789302739.674421:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.674565:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321408 00000100:00100000:2.0:1789302739.674595:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321408 00000100:00100000:2.0:1789302739.674665:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 944 00000100:00100000:2.0:1789302739.674678:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:26487:x1876219350321408:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302739.674704:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302739.674733:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d898 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a041b expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.674820:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.674836:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.674851:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.684728:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.685855:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.685870:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.685950:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.685959:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.685966:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.685988:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302739.685993:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686000:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686013:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302739.686016:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.686021:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686028:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d89f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686040:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686046:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686052:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686057:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302739.686059:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686065:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.686085:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f03927400) 00010000:00010000:2.0:1789302739.686087:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.686091:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.689143:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8a6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.689227:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d898 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a041b expref: 88 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.689241:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302739.689244:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.689249:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184d898 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a041b expref: 88 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.689261:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2a00, rc 0) 00000100:00100000:2.0:1789302739.689344:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:26487:x1876219350321408:12345-192.168.203.11@tcp:101:bash.0 Request processed in 14668us (14938us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302739.689353:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 944 00000100:00100000:1.0:1789302739.700752:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.701027:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321536 00000100:00100000:3.0:1789302739.701067:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321536 00000100:00100000:3.0:1789302739.701146:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 945 00000100:00100000:3.0:1789302739.701158:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+38:26488:x1876219350321536:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302739.701183:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302739.701213:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d8ad lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0422 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.701304:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.701322:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.701361:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302739.701411:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302739.701416:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302739.701439:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.701468:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d8ad lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 40 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.701498:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 39 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.701572:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302739.701575:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302739.701580:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1600, rc 0) 00000100:00100000:3.0:1789302739.701661:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:26488:x1876219350321536:12345-192.168.203.11@tcp:101:rm.0 Request processed in 505us (924us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302739.701670:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 945 00000100:00100000:1.0:1789302739.706183:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.711334:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321664 00000100:00100000:2.0:1789302739.711349:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321664 00000100:00100000:2.0:1789302739.711432:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 946 00000100:00100000:2.0:1789302739.711442:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:26488:x1876219350321664:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302739.711465:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302739.711497:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8bb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0429 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711627:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711640:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711649:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711663:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711668:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711673:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711756:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2800/0x64fe43095184d8d0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711764:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2800/0x64fe43095184d8d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711770:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2800/0x64fe43095184d8d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302739.711793:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 16384 00000004:00200000:2.0:1789302739.711797:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302739.711813:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2800/0x64fe43095184d8d0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711837:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711843:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711849:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711855:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302739.711859:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711866:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711877:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302739.711880:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.711885:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711891:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184d8c2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711903:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711909:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711914:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711922:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302739.711925:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711932:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711937:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xd947e073 (ffff960f03927400) 00010000:00010000:2.0:1789302739.711939:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302739.711941:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711946:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184d8c9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xd947e073 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711975:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184d8bb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0429 expref: 41 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711987:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2800/0x64fe43095184d8d0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0429 expref: 40 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.711993:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302739.711996:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302739.711998:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2800, rc 0) 00000100:00100000:3.0:1789302739.714247:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26488:x1876219350321664:12345-192.168.203.11@tcp:101:rm.0 Request processed in 2790us (8064us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302739.714264:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 946 00000100:00100000:2.0:1789302739.719366:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302739.719873:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321792 00000100:00100000:1.0:1789302739.719884:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321792 00000100:00100000:1.0:1789302739.719940:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 947 00000100:00100000:1.0:1789302739.719950:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26488:x1876219350321792:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302739.719975:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302739.720009:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d8d7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0430 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720277:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720292:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720302:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720315:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720323:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720329:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720380:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720388:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720393:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720400:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302739.720404:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720423:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720435:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302739.720438:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302739.720443:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720450:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184d8de lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720463:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720469:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720486:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.720491:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302739.720494:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.721160:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.721180:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f10d24000) 00010000:00010000:1.0:1789302739.721183:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302739.721187:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.721205:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184d8e5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.721241:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d8d7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0430 expref: 41 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.721250:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302739.721253:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302739.721256:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d8d7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0430 expref: 41 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.721266:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f800, rc 0) 00000100:00100000:2.0:1789302739.726051:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26488:x1876219350321792:12345-192.168.203.11@tcp:101:rm.0 Request processed in 6090us (6683us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302739.726065:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 947 00000100:00100000:1.0:1789302739.729924:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.729990:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350321920 00000100:00100000:3.0:1789302739.729999:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350321920 00000100:00100000:3.0:1789302739.730056:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 948 00000100:00100000:3.0:1789302739.730065:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26488:x1876219350321920:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302739.730101:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302739.730123:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d8ec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0437 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730192:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730205:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730215:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730229:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730235:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730242:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730290:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730296:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730303:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730308:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302739.730317:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730334:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730346:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302739.730350:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302739.730354:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730360:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d8f3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730373:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730379:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730384:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730389:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302739.730392:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730398:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730404:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f0cdf2100) 00010000:00010000:3.0:1789302739.730406:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302739.730414:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730420:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d8fa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730452:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d8ec lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0437 expref: 41 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730462:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302739.730465:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302739.730468:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d8ec lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0437 expref: 41 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.730487:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302739.730798:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26488:x1876219350321920:12345-192.168.203.11@tcp:101:rm.0 Request processed in 732us (881us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302739.730806:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 948 00000100:00100000:1.0:1789302739.761806:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.762809:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350322048 00000100:00100000:3.0:1789302739.762820:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350322048 00000100:00100000:3.0:1789302739.762889:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 949 00000100:00100000:0.0:1789302739.763046:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26489:x1876219350322048:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302739.763066:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302739.763086:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d901 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a043e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763151:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763178:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763190:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763205:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763212:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763218:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763278:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763286:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763292:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763313:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302739.763317:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763324:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763335:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302739.763338:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.763343:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763348:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d908 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763362:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763378:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763384:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763389:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302739.763392:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763398:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763416:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f03501200) 00010000:00010000:0.0:1789302739.763419:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.763422:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763428:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d90f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763461:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d901 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a043e expref: 41 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763481:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302739.763483:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.763487:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d901 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a043e expref: 41 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.763496:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:0.0:1789302739.763598:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26489:x1876219350322048:12345-192.168.203.11@tcp:101:cp.0 Request processed in 699us (1801us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302739.763605:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 949 00000100:00100000:2.0:1789302739.769562:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302739.769714:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350322176 00000100:00100000:0.0:1789302739.769729:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350322176 00000100:00100000:0.0:1789302739.769806:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 950 00000100:00100000:0.0:1789302739.769818:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26489:x1876219350322176:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302739.769846:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302739.769875:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d916 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0445 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.769987:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770005:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770017:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770033:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770039:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770046:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770238:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770247:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770254:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770260:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302739.770265:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770285:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770300:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302739.770302:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.770308:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770314:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d91d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770328:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770334:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770340:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770345:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302739.770348:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770354:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770360:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f03501200) 00010000:00010000:0.0:1789302739.770362:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.770364:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770378:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d924 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770415:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d916 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0445 expref: 41 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770426:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302739.770428:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.770432:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d916 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0445 expref: 41 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.770441:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:0.0:1789302739.770683:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26489:x1876219350322176:12345-192.168.203.11@tcp:101:cp.0 Request processed in 866us (1218us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302739.770691:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 950 00000100:00100000:1.0:1789302739.776665:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.776761:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350322432 00000100:00100000:3.0:1789302739.776774:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350322432 00000100:00100000:3.0:1789302739.776846:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 951 00000100:00100000:3.0:1789302739.776859:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26489:x1876219350322432:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302739.776887:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302739.776915:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d92b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a044c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.777103:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.777120:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.777244:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.777263:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.777271:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.777279:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302739.778957:0:5845:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x14:0x0], index 0: 0 00010000:00010000:3.0:1789302739.781066:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781093:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781108:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781166:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781191:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781199:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781205:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302739.781210:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781219:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781233:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302739.781236:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302739.781241:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781247:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d932 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302739.781263:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a611670e reply state 00000000538b4af1 transno 4294967907 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302739.781273:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302739.781320:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d92b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781336:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.781354:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302739.781357:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302739.781359:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302739.781459:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26489:x1876219350322432:12345-192.168.203.11@tcp:101:cp.0 Request processed in 4602us (4808us total) trans 4294967907 rc 0/0 00000100:00100000:3.0:1789302739.781469:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 951 00010000:00010000:3.0:1789302739.794903:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.794928:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.794936:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.794943:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302739.794949:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.794958:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.794978:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f0cdf2100) 00010000:00010000:3.0:1789302739.794981:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302739.794986:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.794993:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d939 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302739.801550:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.801645:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350322560 00000100:00100000:2.0:1789302739.801657:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350322560 00000100:00100000:2.0:1789302739.801740:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 244 00000100:00100000:2.0:1789302739.801753:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350322560:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302739.801777:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302739.801804:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0453 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302739.801839:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302739.801865:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=576s v=(0 0) 00010000:00010000:2.0:1789302739.801871:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302739.801882:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.801902:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.801917:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2200, rc 0) 00000100:00100000:2.0:1789302739.801994:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350322560:12345-192.168.203.11@tcp:101:cp.0 Request processed in 244us (462us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.802002:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 244 00000100:00080000:0.0:1789302739.802202:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302739.808590:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.808727:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350322688 00000100:00100000:3.0:1789302739.808740:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350322688 00000100:00100000:3.0:1789302739.808812:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 245 00000100:00100000:3.0:1789302739.808825:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:26489:x1876219350322688:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302739.808850:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302739.808875:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a045a expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.808920:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.808930:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302739.808934:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302739.808943:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.808982:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.808997:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 11 pid: 6695 timeout: 676 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302739.809014:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371112064:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302739.809141:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1035b240 going to sleep for 11 seconds 00000100:00100000:1.0:1789302739.811260:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371112064:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302739.811309:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.811334:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:1.0:1789302739.811403:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:26489:x1876219350322688:12345-192.168.203.11@tcp:101:cp.0 Request processed in 2580us (2829us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302739.811412:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 245 00000100:00100000:1.0:1789302739.811846:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302739.811916:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350322816 00000100:00100000:2.0:1789302739.811925:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350322816 00010000:00010000:2.0:1789302739.811962:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 11 pid: 6695 timeout: 676 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302739.812020:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 287 00000100:00100000:2.0:1789302739.812029:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350322816:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302739.812047:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302739.812075:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 11 pid: 6695 timeout: 676 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.812090:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 11 pid: 6695 timeout: 676 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.812116:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006b24994b/0x64fe43095184d94e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0453 expref: 11 pid: 6695 timeout: 676 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.812131:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302739.812136:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302739.812145:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.812156:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302739.812196:0:8463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302739.812212:0:8463:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8463:1876219371112192:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302739.812289:0:8463:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302739.815205:0:8463:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8463:1876219371112192:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302739.815263:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302739.815337:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b1850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350322816:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 3308us (3490us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302739.815350:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 287 00000100:00100000:3.0:1789302739.818629:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302739.818983:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350323072 00000100:00100000:1.0:1789302739.819013:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350323072 00000100:00100000:1.0:1789302739.819089:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 190 00000100:00100000:1.0:1789302739.819101:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ee850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26489:x1876219350323072:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:1.0:1789302739.820048:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ee850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26489:x1876219350323072:12345-192.168.203.11@tcp:35:cp.0 Request processed in 935us (1422us total) trans 4294967908 rc 0/0 00000100:00100000:1.0:1789302739.820060:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 190 00000100:00100000:1.0:1789302739.860627:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302739.860845:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350323200 00000100:00100000:0.0:1789302739.860859:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350323200 00000100:00100000:0.0:1789302739.860938:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 952 00000100:00100000:0.0:1789302739.860951:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:26492:x1876219350323200:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302739.860978:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302739.861005:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d95c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0461 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861098:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861151:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861167:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861185:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861192:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861198:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861306:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861316:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861324:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302739.861438:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00000004:00200000:0.0:1789302739.861444:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302739.861467:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861489:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861682:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861690:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861701:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861824:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302739.861829:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861838:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861852:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302739.861855:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.861861:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861868:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184d963 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861883:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861889:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861895:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861908:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302739.861911:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861917:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861925:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f03501200) 00010000:00010000:0.0:1789302739.861928:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302739.861931:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861937:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184d96a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861975:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184d95c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 89 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861991:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.861999:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302739.862001:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302739.862004:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302739.862088:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:26492:x1876219350323200:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1138us (1474us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302739.862097:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 952 00000100:00100000:2.0:1789302739.870961:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302739.871052:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350323328 00000100:00100000:0.0:1789302739.871084:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350323328 00000100:00100000:0.0:1789302739.871186:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 246 00000100:00100000:0.0:1789302739.871198:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350323328:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302739.871222:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302739.871252:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184d978 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0468 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302739.871348:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7ca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371112320:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302739.871495:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf6c0 going to sleep for 11 seconds 00000100:00080000:2.0:1789302739.873096:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789302739.879764:0:8571:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1789302739.879776:0:8571:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb mtime: 0 -> 1789302739 00002000:00010000:0.0:1789302739.879780:0:8571:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb atime: 0 -> 1789302739 00002000:00010000:0.0:1789302739.879782:0:8571:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x14:0x0] updating lvb ctime: 0 -> 1789302739 00000100:00100000:0.0:1789302739.879814:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7ca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371112320:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302739.879859:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184d978 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0468 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.879889:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184d978 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0468 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302739.879905:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:0.0:1789302739.879971:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350323328:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 8776us (9025us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302739.879981:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 246 00000100:00100000:0.0:1789302739.911697:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.911960:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350323456 00000100:00100000:3.0:1789302739.911974:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350323456 00000100:00100000:3.0:1789302739.912051:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 953 00000100:00100000:3.0:1789302739.912065:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:26492:x1876219350323456:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302739.912092:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302739.912121:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d97f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a046f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.912228:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.912245:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.912271:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302739.912317:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3c:0x0]: returning size 16384 00000004:00200000:3.0:1789302739.912320:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302739.912342:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.912374:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d97f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 90 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.912420:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 89 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302739.912429:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:3.0:1789302739.912433:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302739.912438:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1400, rc 0) 00000100:00100000:3.0:1789302739.912996:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:26492:x1876219350323456:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 930us (1310us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302739.913006:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 953 00000100:00100000:2.0:1789302739.919843:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302739.920023:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350323584 00000100:00100000:1.0:1789302739.920037:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350323584 00000100:00100000:1.0:1789302739.920113:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 247 00000100:00100000:1.0:1789302739.920126:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350323584:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302739.920152:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302739.920178:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d98d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0476 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302739.920259:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371112448:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302739.920380:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302739.927808:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371112448:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302739.927864:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d98d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0476 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.927894:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d98d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0476 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302739.927911:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120ddc00, rc 0) 00000100:00100000:1.0:1789302739.927999:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350323584:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 7876us (8171us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302739.928009:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 247 00000100:00100000:0.0:1789302739.945036:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302739.946715:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350323840 00000100:00100000:3.0:1789302739.946742:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350323840 00000100:00100000:3.0:1789302739.946826:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 954 00000100:00100000:3.0:1789302739.946839:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:26492:x1876219350323840:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302739.947218:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].24, 136 off11456 00000100:00100000:3.0:1789302739.947999:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:26492:x1876219350323840:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 1158us (2972us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302739.948011:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 954 00000040:00080000:1.0:1789302739.980867:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302739.980881:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302739.980929:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 25 count 3 00000040:00080000:1.0:1789302739.980939:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302739.981291:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371112576:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302739.981426:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302739.982425:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371112960:0@lo:400:kworker.0 00000100:00100000:3.0:1789302739.982539:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302739.982585:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789302739.982591:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371112832:0@lo:400:kworker.0 00000100:00100000:3.0:1789302739.982616:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302739.982632:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371113344:0@lo:400:kworker.0 00000100:00100000:3.0:1789302739.982661:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302739.982859:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371113088:0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.982894:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302739.982926:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1789302739.982931:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371112704:0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.982961:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302739.982990:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371113216:0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.983013:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302739.983044:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371112704 00000100:00100000:2.0:1789302739.983054:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371112704 00000100:00100000:2.0:1789302739.983106:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 550 00000100:00100000:2.0:1789302739.983116:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371112704:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.983176:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371112704:12345-0@lo:400:kworker.0 Request processed in 61us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.983184:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 550 00000100:00100000:2.0:1789302739.983230:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371113344 00000100:00100000:2.0:1789302739.983234:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371113344 00000100:00100000:2.0:1789302739.983273:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371113088 00000100:00100000:2.0:1789302739.983276:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371113088 00000100:00100000:2.0:1789302739.983300:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1789302739.983306:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371113344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.983346:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371113344:12345-0@lo:400:kworker.0 Request processed in 40us (684us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.983352:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1789302739.983376:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 361 00000100:00100000:2.0:1789302739.983382:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371113088:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.983415:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371113088:12345-0@lo:400:kworker.0 Request processed in 33us (521us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.983421:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 361 00000100:00100000:2.0:1789302739.983463:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371113344:0@lo:400:kworker.0 00000100:00080000:2.0:1789302739.983547:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302739.983554:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302739.983570:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371112704:0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.983580:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371113088:0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.983600:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371112960 00000100:00100000:2.0:1789302739.983603:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371112960 00000100:00100000:2.0:1789302739.983634:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371112832 00000100:00100000:2.0:1789302739.983637:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371112832 00000100:00100000:2.0:1789302739.983651:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371113216 00000100:00100000:2.0:1789302739.983654:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371113216 00000100:00100000:2.0:1789302739.983667:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 372 00000100:00100000:2.0:1789302739.983673:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371112960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.983712:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371112960:12345-0@lo:400:kworker.0 Request processed in 38us (1173us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.983976:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 372 00000100:00100000:2.0:1789302739.983991:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789302739.983998:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371112832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.984032:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371112832:12345-0@lo:400:kworker.0 Request processed in 35us (1415us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.984038:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 373 00000100:00100000:2.0:1789302739.984060:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1789302739.984066:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371113216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.984103:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371113216:12345-0@lo:400:kworker.0 Request processed in 38us (1089us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302739.984108:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 374 00000100:00100000:2.0:1789302739.984139:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371112832:0@lo:400:kworker.0 00000100:00100000:2.0:1789302739.984150:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371112960:0@lo:400:kworker.0 00000100:00080000:2.0:1789302739.984167:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302739.984182:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371113216:0@lo:400:kworker.0 00000100:00100000:0.0:1789302740.030927:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302740.031044:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0700 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371112576:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302740.031244:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350323968 00000100:00100000:2.0:1789302740.031269:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350323968 00000100:00100000:2.0:1789302740.031352:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 955 00000100:00100000:2.0:1789302740.031363:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26495:x1876219350323968:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302740.031407:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302740.031595:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26495:x1876219350323968:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 233us (675us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302740.031603:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 955 00000100:00100000:2.0:1789302740.044852:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302740.044960:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350324224 00000100:00100000:0.0:1789302740.044975:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350324224 00000100:00100000:0.0:1789302740.045036:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 956 00000100:00100000:0.0:1789302740.045046:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26495:x1876219350324224:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302740.045070:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302740.045092:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184d994 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a047d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.045194:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184d99b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.045221:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184d99b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.045237:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184d99b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302740.045248:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:0.0:1789302740.045407:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184d994 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a047d expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.045427:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184d99b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a047d expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.045436:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302740.045439:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302740.045443:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517800, rc 0) 00000100:00100000:0.0:1789302740.045572:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26495:x1876219350324224:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 526us (737us total) trans 4294967909 rc 0/0 00000100:00100000:0.0:1789302740.045595:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 956 00000100:00100000:0.0:1789302740.073908:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.075120:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302740.075755:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350324480 00000100:00100000:0.0:1789302740.075765:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350324480 00000100:00100000:0.0:1789302740.075835:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 41 00000100:00100000:0.0:1789302740.075845:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2345:x1876219350324480:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302740.075860:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302740.075884:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184d99b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a047d expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.075904:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302740.075909:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302740.075967:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2345:x1876219350324480:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 123us (850us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302740.075975:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 41 00000100:00100000:2.0:1789302740.076552:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350324352 00000100:00100000:2.0:1789302740.076566:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350324352 00000100:00100000:2.0:1789302740.076638:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 248 00000100:00100000:2.0:1789302740.076648:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350324352:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302740.076699:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.076754:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.076764:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789302740.076769:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302740.076774:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.076815:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.076833:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 10 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302740.076852:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371113472:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302740.076964:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b180 going to sleep for 11 seconds 00000100:00100000:2.0:1789302740.081693:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371113472:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302740.081753:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302740.081834:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302740.081919:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350324608 00000100:00100000:2.0:1789302740.081929:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350324608 00002000:00100000:2.0:1789302740.081992:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x14:0x0] (0->157) req@ffff960f3a9e0700 x1876219350324608/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:316/0 lens 648/0 e 0 to 0 dl 1789302751 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302740.082032:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3a9e0700 with x1876219350324608 ext(0->157) 00010000:00010000:2.0:1789302740.082043:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 11 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.082059:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 11 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302740.082068:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3a9e0700 00000100:00100000:2.0:1789302740.082115:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 7 00000100:00100000:2.0:1789302740.082127:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350324608:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:2.0:1789302740.082969:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350324608:12345-192.168.203.11@tcp:4:cp.0 Request processed in 841us (1139us total) trans 4294967345 rc 0/0 00000100:00100000:2.0:1789302740.082979:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 7 00002000:00100000:2.0:1789302740.082990:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x14:0x0] (0->157) req@ffff960f3a9e0700 x1876219350324608/t4294967345(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:316/0 lens 648/448 e 0 to 0 dl 1789302751 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302740.083000:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3a9e0700 with x1876219350324608 ext(0->157) 00010000:00010000:2.0:1789302740.083008:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 13 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.083028:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 13 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302740.083036:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3a9e0700 00000100:00100000:1.0:1789302740.089874:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.090047:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350324736 00000100:00100000:3.0:1789302740.090057:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350324736 00010000:00010000:3.0:1789302740.090093:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 13 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302740.090209:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:3.0:1789302740.090220:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2346:x1876219350324736:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302740.090236:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302740.090261:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 13 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.090286:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 13 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.090310:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004c7ddea/0x64fe43095184d955 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a045a expref: 13 pid: 13439 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.090329:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302740.090332:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302740.090338:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.090348:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.090386:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302740.090463:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350324736:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 244us (597us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302740.090480:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 288 00010000:00010000:2.0:1789302740.091075:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091093:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091119:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091138:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091146:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091153:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302740.091156:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091164:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091178:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.091184:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9a9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302740.091259:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350324352:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 14611us (17359us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302740.092899:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 248 00000100:00100000:1.0:1789302740.098092:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.098201:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350324992 00000100:00100000:3.0:1789302740.098212:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350324992 00000100:00100000:3.0:1789302740.098305:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 957 00000100:00100000:3.0:1789302740.098316:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:26495:x1876219350324992:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302740.098433:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:26495:x1876219350324992:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 113us (345us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302740.098441:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 957 00000100:00100000:1.0:1789302740.109823:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302740.109924:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350325120 00000100:00100000:2.0:1789302740.109936:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350325120 00000100:00100000:2.0:1789302740.109991:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:2.0:1789302740.110001:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350325120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302740.110020:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302740.110041:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0484 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302740.110070:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 158, mtime: 0x6aa697d3, atime: 0x0, ctime: 0x6aa697d3, blocks: 0x1 00010000:00010000:2.0:1789302740.110090:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=577s v=(0 0) 00010000:00010000:2.0:1789302740.110096:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302740.110105:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.110135:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302740.110150:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2a00, rc 0) 00000100:00100000:2.0:1789302740.110213:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350325120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 213us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302740.110220:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:1.0:1789302740.128379:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302740.128536:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350325376 00000100:00100000:2.0:1789302740.128546:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350325376 00000100:00100000:2.0:1789302740.128594:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 958 00000100:00100000:2.0:1789302740.128603:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:26495:x1876219350325376:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302740.128714:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:26495:x1876219350325376:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 112us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302740.128724:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 958 00000100:00100000:1.0:1789302740.140311:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302740.140537:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350325504 00000100:00100000:0.0:1789302740.140549:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350325504 00002000:00100000:0.0:1789302740.140595:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x14:0x0] (0->4095) req@ffff960f3ffa4380 x1876219350325504/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:316/0 lens 488/0 e 0 to 0 dl 1789302751 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302740.140607:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffa4380 with x1876219350325504 ext(0->4095) 00002000:00010000:0.0:1789302740.140613:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ffa4380 00000100:00100000:0.0:1789302740.140653:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 92 00000100:00100000:0.0:1789302740.140664:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350325504:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302740.140861:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350325504:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 197us (557us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302740.140883:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 92 00002000:00100000:0.0:1789302740.140890:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x14:0x0] (0->4095) req@ffff960f3ffa4380 x1876219350325504/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:316/0 lens 488/4536 e 0 to 0 dl 1789302751 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302740.140897:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffa4380 with x1876219350325504 ext(0->4095) 00002000:00010000:0.0:1789302740.140901:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ffa4380 00000100:00100000:1.0:1789302740.149896:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.149990:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350325632 00000100:00100000:3.0:1789302740.150001:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350325632 00000100:00100000:3.0:1789302740.150061:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 959 00000100:00100000:3.0:1789302740.150072:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:26495:x1876219350325632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302740.150092:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302740.150114:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d9b7 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a048b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.150149:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.150163:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.150173:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.150283:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184d9b7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.150310:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.150318:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302740.150321:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302740.150324:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302740.150389:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26495:x1876219350325632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 320us (503us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302740.150395:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 959 00000100:00100000:1.0:1789302740.160026:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.160110:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350325760 00000100:00100000:3.0:1789302740.160120:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350325760 00000100:00100000:3.0:1789302740.160185:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 960 00000100:00100000:3.0:1789302740.160196:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26495:x1876219350325760:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302740.160233:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302740.160328:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26495:x1876219350325760:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 134us (312us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302740.160335:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 960 00000100:00100000:3.0:1789302740.167719:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302740.167857:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350325888 00000100:00100000:0.0:1789302740.167867:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350325888 00000100:00100000:0.0:1789302740.167923:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 250 00000100:00100000:0.0:1789302740.167934:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350325888:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302740.167976:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.167992:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302740.167996:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302740.168001:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168016:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168043:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168051:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168072:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168079:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302740.168084:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168093:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168108:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302740.168116:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9c5 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302740.168187:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350325888:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 254us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302740.168195:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 250 00000100:00100000:0.0:1789302740.175777:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.175922:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350326144 00000100:00100000:3.0:1789302740.175932:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350326144 00000100:00100000:3.0:1789302740.175993:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 961 00000100:00100000:3.0:1789302740.176003:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26495:x1876219350326144:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302740.176078:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.176114:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.176123:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 4/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.176159:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302740.176173:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 45 pid: 5845 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302740.176187:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371113600:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302740.176267:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1035b840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302740.180919:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.181267:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371113600:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302740.181294:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302740.182712:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350326272 00000100:00100000:1.0:1789302740.182723:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350326272 00010000:00010000:1.0:1789302740.182769:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 46 pid: 5845 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302740.182809:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 289 00000100:00100000:1.0:1789302740.182819:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2348:x1876219350326272:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302740.182834:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302740.182856:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 46 pid: 5845 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.182867:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 46 pid: 5845 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.182888:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184d9be lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a048b expref: 46 pid: 5845 timeout: 677 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.182912:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302740.182918:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.182927:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.182935:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.182960:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302740.183011:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2348:x1876219350326272:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 194us (2107us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302740.183019:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 289 00010000:00010000:1.0:1789302740.183045:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.183053:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.184398:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.184422:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.184440:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.184447:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302740.184451:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.184460:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.184482:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302740.184485:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302740.184489:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 2/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302740.184496:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184d9cc lrc: 0/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302740.186016:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26495:x1876219350326144:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 10009us (10245us total) trans 4294967910 rc 0/0 00000100:00100000:1.0:1789302740.186025:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 961 00000040:00080000:1.0:1789302740.191347:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302740.191358:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302740.191969:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1789302740.191979:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302740.192896:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.193078:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350326528 00000100:00100000:3.0:1789302740.193096:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350326528 00000100:00100000:3.0:1789302740.193147:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 191 00000100:00100000:3.0:1789302740.193157:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26495:x1876219350326528:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302740.193396:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26495:x1876219350326528:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 240us (504us total) trans 4294967911 rc 0/0 00000100:00100000:3.0:1789302740.193405:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 191 00000100:00100000:1.0:1789302740.241519:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302740.241642:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350326656 00000100:00100000:3.0:1789302740.241659:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350326656 00000100:00100000:3.0:1789302740.241717:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 551 00000100:00100000:3.0:1789302740.241728:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350326656:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302740.241809:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350326656:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 83us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302740.241819:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 551 00000100:00100000:1.0:1789302740.242354:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302740.242389:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302740.242451:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350326912 00000100:00100000:2.0:1789302740.242457:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350326912 00000100:00100000:2.0:1789302740.242537:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350326784 00000100:00100000:2.0:1789302740.242549:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350326784 00000100:00100000:2.0:1789302740.242574:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:2.0:1789302740.242582:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350326912:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302740.242625:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350326912:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 45us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302740.242631:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:2.0:1789302740.242643:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 363 00000100:00100000:2.0:1789302740.242649:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2346:x1876219350326784:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302740.242673:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2346:x1876219350326784:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302740.242678:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 363 00000100:00080000:1.0:1789302740.243285:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302740.501830:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789302740.501851:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371113728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302740.501918:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302740.501997:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371113728 00000100:00100000:0.0:1789302740.502005:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371113728 00000100:00100000:0.0:1789302740.502046:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1789302740.502055:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371113728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302740.502086:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 73072640 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302740.502148:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371113728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 94us (229us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302740.502155:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 160 00000100:00100000:0.0:1789302740.502198:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371113728:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789302740.621941:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967909 is committed 00000001:00080000:2.0:1789302740.621959:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967910 is committed 00000001:00080000:2.0:1789302740.621971:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967906 is committed 00000001:00080000:2.0:1789302740.621976:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967911 is committed 00010000:00010000:2.0:1789302740.637304:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302741.018584:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371113856:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302741.018650:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302741.018724:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371113856 00000100:00100000:0.0:1789302741.018731:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371113856 00000100:00100000:0.0:1789302741.018785:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:0.0:1789302741.018795:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371113856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302741.018816:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 76480512 pending 1703936 free 3857711104 avail 3855613952 00000100:00100000:0.0:1789302741.018859:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371113856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302741.018866:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000100:00080000:0.0:1789302741.018886:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302741.018919:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371113856:0@lo:13:osp-pre-0-0.0 00000040:00080000:1.0:1789302741.155408:0:19271:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302741.155416:0:19271:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302741.155480:0:19271:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 25 count 3 00000040:00080000:1.0:1789302741.155486:0:19271:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302741.220592:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302741.220707:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350327040 00000100:00100000:1.0:1789302741.220722:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350327040 00000100:00100000:1.0:1789302741.220823:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 251 00000100:00100000:1.0:1789302741.220834:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350327040:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302741.221615:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302741.221637:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0492 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.221666:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302741.221670:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302741.221680:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.221698:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.221711:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120ddc00, rc 0) 00000100:00100000:1.0:1789302741.221773:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350327040:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 941us (1196us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302741.221800:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 251 00000100:00080000:2.0:1789302741.222617:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302741.245749:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302741.245940:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350327296 00000100:00100000:1.0:1789302741.245950:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350327296 00000100:00100000:1.0:1789302741.246022:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 962 00000100:00100000:1.0:1789302741.246032:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:26503:x1876219350327296:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302741.246051:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302741.246074:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d9da lrc: 2/0,0 mode: --/CR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0499 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.246178:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9e1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.246193:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.246205:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9e1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302741.246217:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:1.0:1789302741.246382:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184d9da lrc: 0/0,0 mode: --/CR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0499 expref: 91 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.246397:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9e1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0499 expref: 90 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.246406:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302741.246409:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1789302741.246413:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f800, rc 0) 00000100:00100000:1.0:1789302741.246482:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:26503:x1876219350327296:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 451us (745us total) trans 4294967912 rc 0/0 00000100:00100000:1.0:1789302741.246489:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 962 00000100:00080000:0.0:1789302741.247000:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302741.250884:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302741.250964:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350327424 00000100:00100000:0.0:1789302741.250976:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350327424 00000100:00100000:0.0:1789302741.251037:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 252 00000100:00100000:0.0:1789302741.251047:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350327424:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:0.0:1789302741.251114:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.251135:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.251144:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.251152:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302741.251155:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302741.251169:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 5/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.251208:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.251223:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 12 pid: 8571 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302741.251238:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371113984:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302741.251334:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.251344:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 11 pid: 6697 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302741.251361:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371114112:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302741.251388:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cfe40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302741.267543:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302741.268403:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350327552 00000100:00100000:2.0:1789302741.268414:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350327552 00010000:00010000:2.0:1789302741.268488:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 12 pid: 6697 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302741.268537:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 290 00000100:00100000:2.0:1789302741.268546:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350327552:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302741.268559:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302741.268580:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 12 pid: 6697 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302741.268593:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 12 pid: 6697 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302741.268617:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302741.268681:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350327552:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 136us (1148us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302741.268688:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 290 00000100:00080000:1.0:1789302741.272314:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302741.277646:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302741.277685:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350327680 00000100:00100000:1.0:1789302741.277694:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350327680 00010000:00010000:1.0:1789302741.277733:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 13 pid: 8571 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302741.277777:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 291 00000100:00100000:1.0:1789302741.277786:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350327680:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302741.277801:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302741.277823:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 13 pid: 8571 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.277836:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x14:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 13 pid: 8571 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.277861:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302741.277865:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302741.277871:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.277880:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.277910:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302741.277964:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350327680:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 178us (326us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302741.277971:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 291 00010000:00010000:0.0:1789302741.280043:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003be8e167/0x64fe43095184d9d3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0492 expref: 12 pid: 8571 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302741.280062:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371113984:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302741.280150:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184d9b0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0484 expref: 11 pid: 6697 timeout: 678 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302741.280161:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371114112:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302741.280186:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280197:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280205:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280237:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280247:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280266:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280274:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302741.280278:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280288:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280306:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.280314:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000041460e53/0x64fe43095184d9ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302741.280398:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350327424:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 29351us (29521us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302741.280406:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 252 00000100:00100000:1.0:1789302741.286011:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302741.286150:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350327936 00000100:00100000:3.0:1789302741.286170:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350327936 00000100:00100000:3.0:1789302741.286253:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 192 00000100:00100000:3.0:1789302741.286265:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:26503:x1876219350327936:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:3.0:1789302741.286352:0:5847:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x19:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9e1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0499 expref: 90 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302741.286902:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967913 00000001:00080000:3.0:1789302741.289581:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967914 00010000:00010000:3.0:1789302741.289705:0:5847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.289733:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.289743:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.289753:0:5847:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.289818:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.289838:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 93 pid: 5844 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302741.289858:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371114240:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302741.289999:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.290009:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 40 pid: 5845 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302741.290019:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371114368:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302741.290044:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3300 going to sleep for 11 seconds 00000100:00100000:3.0:1789302741.293019:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302741.293059:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371114240:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302741.293078:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371114368:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302741.293125:0:5847:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302741.294282:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302741.294443:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350328192 00000100:00100000:1.0:1789302741.294457:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350328192 00010000:00010000:1.0:1789302741.294558:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 41 pid: 5845 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302741.294591:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350328064 00000100:00100000:1.0:1789302741.294594:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350328064 00010000:00010000:1.0:1789302741.294609:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 94 pid: 5844 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302741.294628:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00100000:1.0:1789302741.294638:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2345:x1876219350328192:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302741.294652:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302741.294658:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 41 pid: 5845 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.294691:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 41 pid: 5845 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.294714:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184d940 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a044c expref: 41 pid: 5845 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.294745:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302741.294751:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.294760:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.294766:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302741.294895:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:0.0:1789302741.294917:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:2345:x1876219350328064:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302741.294939:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302741.294950:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 94 pid: 5844 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.294986:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 94 pid: 5844 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.295008:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184d971 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0461 expref: 94 pid: 5844 timeout: 678 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.295026:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302741.295043:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.295055:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.295064:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.295100:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302741.295123:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:2345:x1876219350328192:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 482us (2118us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302741.295133:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00100000:0.0:1789302741.295154:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b0450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:2345:x1876219350328064:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 243us (878us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302741.295164:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 293 00010000:00010000:1.0:1789302741.295292:0:5847:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.295302:0:5847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789302741.295428:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967915 00000001:00080000:1.0:1789302741.295673:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967916 00000001:00080000:1.0:1789302741.296239:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967917 00000100:00080000:3.0:1789302741.296295:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302741.296346:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371114496:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1789302741.296436:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302741.296494:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335896c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302741.296554:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371114496 00000100:00100000:3.0:1789302741.296566:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371114496 00000100:00100000:3.0:1789302741.296639:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 33 00000100:00100000:3.0:1789302741.296652:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371114496:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1789302741.297164:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371114496:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 514us (725us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302741.297173:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 33 00000100:00100000:0.0:1789302741.297331:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371114496:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:0.0:1789302741.298043:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967918 00000100:00100000:0.0:1789302741.298143:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02908380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371114624:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1789302741.298199:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302741.298237:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf300 going to sleep for 11 seconds 00000100:00100000:3.0:1789302741.298892:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371114624 00000100:00100000:3.0:1789302741.298903:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371114624 00000100:00100000:3.0:1789302741.298961:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 34 00000100:00100000:3.0:1789302741.298971:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371114624:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1789302741.299281:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5847:x1876219371114624:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 310us (1081us total) trans 4294967346 rc 0/0 00000100:00100000:3.0:1789302741.299289:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 34 00000100:00100000:0.0:1789302741.299402:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02908380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371114624:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:0.0:1789302741.299447:0:5847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.299457:0:5847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.299476:0:5847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.299483:0:5847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302741.299489:0:5847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.299498:0:5847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.299641:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302741.299646:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1789302741.299652:0:5847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.299716:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184d9f6 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302741.299972:0:5847:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x3d:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1789302741.303896:0:5847:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].11, 64 off8920 00000004:00080000:0.0:1789302741.303928:0:5847:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 578314424483ns, next at 579314418218ns, rc = 0 00000001:00080000:0.0:1789302741.303950:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967919 00010000:00010000:0.0:1789302741.304049:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302741.304054:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302741.304069:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9e1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0499 expref: 97 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302741.304259:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:26503:x1876219350327936:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 17995us (18256us total) trans 4294967920 rc 0/0 00000100:00100000:0.0:1789302741.304272:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 192 00000020:00080000:0.0:1789302741.304279:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:3.0:1789302741.331185:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302741.331284:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350328320 00000100:00100000:1.0:1789302741.331297:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350328320 00000100:00100000:1.0:1789302741.331376:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 963 00000100:00100000:1.0:1789302741.331388:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:26504:x1876219350328320:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302741.331422:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302741.331454:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04a0 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.331584:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.331604:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.331619:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.331635:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.331643:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.331650:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332031:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184da12 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332041:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184da12 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332061:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184da12 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302741.332108:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00000004:00200000:1.0:1789302741.332113:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302741.332136:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184da12 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332149:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184da12 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332157:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332164:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332171:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332178:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302741.332183:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332193:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332207:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302741.332210:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302741.332216:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332234:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184da04 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332247:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332253:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332260:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332266:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302741.332269:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332275:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332283:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f10d24300) 00010000:00010000:1.0:1789302741.332285:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302741.332288:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332294:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184da0b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332327:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184d9fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04a0 expref: 98 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332352:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184da12 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04a0 expref: 97 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302741.332361:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302741.332364:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302741.332367:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc 0) 00000100:00100000:1.0:1789302741.332456:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:26504:x1876219350328320:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1071us (1290us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302741.332474:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 963 00000100:00100000:3.0:1789302741.351924:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302741.352145:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350328576 00000100:00100000:0.0:1789302741.352155:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350328576 00000100:00100000:0.0:1789302741.352234:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 42 00000100:00100000:0.0:1789302741.352243:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:26504:x1876219350328576:12345-192.168.203.11@tcp:103:lfs.0 00010000:00010000:0.0:1789302741.352258:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302741.352282:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184da12 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04a0 expref: 97 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302741.352302:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:0.0:1789302741.352312:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302741.352363:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:26504:x1876219350328576:12345-192.168.203.11@tcp:103:lfs.0 Request processed in 120us (450us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302741.352370:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 42 00000100:00100000:2.0:1789302741.355706:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302741.355799:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350328704 00000100:00100000:3.0:1789302741.355810:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350328704 00000100:00100000:3.0:1789302741.355866:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 964 00000100:00100000:3.0:1789302741.355876:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:26504:x1876219350328704:12345-192.168.203.11@tcp:58:lfs.0 00010000:00010000:3.0:1789302741.356352:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.356370:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302741.356383:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302741.356641:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].25, 136 off11592 00000100:00100000:3.0:1789302741.356741:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:26504:x1876219350328704:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 866us (1043us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302741.356749:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 964 00010000:00010000:1.0:1789302741.646639:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302742.221284:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302742.221294:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302742.221339:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 26 count 4 00000040:00080000:1.0:1789302742.221346:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302742.222933:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371114752:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302742.223035:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58a3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302742.248570:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.248644:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371114752:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302742.248791:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350328832 00000100:00100000:2.0:1789302742.248800:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350328832 00000100:00100000:2.0:1789302742.248852:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 965 00000100:00100000:2.0:1789302742.248861:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26514:x1876219350328832:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302742.248905:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302742.248999:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26514:x1876219350328832:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 138us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302742.249006:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 965 00000100:00100000:3.0:1789302742.254113:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302742.254152:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350328960 00000100:00100000:3.0:1789302742.254161:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350328960 00000100:00100000:3.0:1789302742.254229:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 966 00000100:00100000:3.0:1789302742.254240:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26514:x1876219350328960:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1789302742.254311:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254328:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254342:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254358:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254364:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254371:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254397:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254406:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254413:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302742.254450:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00000004:00200000:3.0:1789302742.254464:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302742.254476:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254494:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254500:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254528:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254533:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302742.254537:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254543:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254555:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302742.254558:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302742.254562:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254568:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184da20 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254581:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254586:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254592:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254614:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302742.254618:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254624:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254632:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x8876c877 (ffff960f0cdf2d00) 00010000:00010000:3.0:1789302742.254635:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302742.254637:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254643:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da27 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8876c877 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254657:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254663:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254668:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254672:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302742.254674:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254679:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254695:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2100) 00010000:00010000:3.0:1789302742.254697:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302742.254700:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.254706:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da2e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302742.254775:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26514:x1876219350328960:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 536us (670us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302742.254783:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 966 00000040:00080000:1.0:1789302742.256888:0:19323:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302742.256897:0:19323:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302742.256945:0:19323:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 26 count 4 00000040:00080000:1.0:1789302742.256951:0:19323:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302742.263473:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.263527:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350329216 00000100:00100000:0.0:1789302742.263536:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350329216 00000100:00100000:0.0:1789302742.263589:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 967 00000100:00100000:0.0:1789302742.263599:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26514:x1876219350329216:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302742.263631:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302742.263652:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184da35 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04a7 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263759:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263774:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263785:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302742.263797:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:0.0:1789302742.263896:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263905:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263911:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263917:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302742.263922:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263930:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263940:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2100) 00010000:00010000:0.0:1789302742.263943:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302742.263948:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.263962:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184da3c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.264000:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184da35 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04a7 expref: 42 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.264009:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302742.264011:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302742.264014:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184da35 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04a7 expref: 42 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.264022:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc 0) 00000100:00100000:0.0:1789302742.264078:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:26514:x1876219350329216:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 479us (623us total) trans 4294967921 rc 301/301 00000100:00100000:0.0:1789302742.264085:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 967 00000100:00100000:1.0:1789302742.272808:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302742.273312:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350329344 00000100:00100000:3.0:1789302742.273322:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350329344 00000100:00100000:3.0:1789302742.273378:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 968 00000100:00100000:3.0:1789302742.273388:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:26514:x1876219350329344:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302742.273422:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x19:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302742.273548:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:26514:x1876219350329344:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 160us (748us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302742.273555:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 968 00000100:00100000:0.0:1789302742.279451:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.283294:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350329600 00000100:00100000:2.0:1789302742.283309:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350329600 00000100:00100000:3.0:1789302742.284610:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 969 00000100:00100000:3.0:1789302742.284623:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:26514:x1876219350329600:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302742.284644:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302742.284675:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da4a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04ae expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.285357:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da51 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.285381:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da51 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.285393:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da51 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302742.285404:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 0 00010000:00010000:3.0:1789302742.285462:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da4a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ae expref: 44 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.285482:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da51 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ae expref: 43 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.285490:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f0cdf2d00) 00010000:00010000:3.0:1789302742.285493:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302742.285496:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302742.285587:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26514:x1876219350329600:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 967us (6143us total) trans 4294967922 rc 0/0 00000100:00100000:3.0:1789302742.285594:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 969 00000100:00100000:0.0:1789302742.298834:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.299635:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350329856 00000100:00100000:2.0:1789302742.299647:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350329856 00000100:00100000:2.0:1789302742.299703:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 970 00000100:00100000:2.0:1789302742.299713:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26514:x1876219350329856:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302742.299767:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302742.299792:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184da51 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ae expref: 43 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.299813:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f0cdf2d00) 00010000:00010000:2.0:1789302742.299823:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302742.299845:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.299859:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 95 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.299867:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 4/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.299901:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 95 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.299935:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 95 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302742.299950:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371114880:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302742.300040:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b000 going to sleep for 11 seconds 00000100:00100000:1.0:1789302742.310884:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371114880:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302742.310918:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302742.310990:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.311345:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350329984 00000100:00100000:1.0:1789302742.311363:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350329984 00010000:00010000:1.0:1789302742.311406:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 96 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.311463:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 294 00000100:00100000:1.0:1789302742.311472:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350329984:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302742.311486:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302742.311494:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 96 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.311782:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 96 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.311804:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184d986 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a046f expref: 96 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.311819:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:1.0:1789302742.311824:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.311833:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.311840:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.311866:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302742.311919:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:2347:x1876219350329984:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 448us (929us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302742.311937:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 294 00010000:00010000:2.0:1789302742.314580:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.314814:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.314959:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.314973:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.314983:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315174:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315182:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315259:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315266:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302742.315277:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315285:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315299:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:2.0:1789302742.315303:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302742.315308:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315315:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184da5f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315324:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315331:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315336:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315341:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302742.315354:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315360:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315367:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0xec5ff6a0 (ffff960f03927100) 00010000:00010000:2.0:1789302742.315369:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302742.315377:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.315383:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184da66 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3c:0x0].0xec5ff6a0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302742.315487:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26514:x1876219350329856:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 15771us (16659us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302742.315496:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 970 00000100:00100000:0.0:1789302742.328770:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.328871:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350330112 00000100:00100000:2.0:1789302742.328882:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350330112 00000100:00100000:2.0:1789302742.328929:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 971 00000100:00100000:2.0:1789302742.328938:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26514:x1876219350330112:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302742.328958:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302742.328981:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da6d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04b5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.329016:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.329028:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.329038:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.329142:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da6d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 44 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.329159:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.329174:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:2.0:1789302742.329177:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302742.329180:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2c00, rc 0) 00000100:00100000:2.0:1789302742.329234:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26514:x1876219350330112:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 296us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302742.329240:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 971 00000100:00100000:0.0:1789302742.345938:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302742.346052:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350330368 00000100:00100000:3.0:1789302742.346062:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350330368 00000100:00100000:3.0:1789302742.346122:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 972 00000100:00100000:3.0:1789302742.346133:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26514:x1876219350330368:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302742.346211:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.346230:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.346256:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302742.346446:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000080daa442 reply state 00000000d5dd58b2 transno 4294967923 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302742.346467:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302742.346576:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26514:x1876219350330368:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 445us (647us total) trans 4294967923 rc 0/0 00000100:00100000:3.0:1789302742.346584:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 972 00000100:00100000:2.0:1789302742.353245:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.353283:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350330496 00000100:00100000:2.0:1789302742.353291:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350330496 00000100:00100000:2.0:1789302742.353338:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 973 00000100:00100000:2.0:1789302742.353349:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26514:x1876219350330496:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302742.353372:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302742.353391:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04bc expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.353446:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.353472:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.353482:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:2.0:1789302742.353485:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302742.353489:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3e00, rc 0) 00000100:00100000:2.0:1789302742.353579:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26514:x1876219350330496:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 228us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302742.353595:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 973 00010000:00010000:3.0:1789302742.354238:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.354249:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.354256:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.354262:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302742.354267:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.354274:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.354286:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302742.354288:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789302742.354293:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.354312:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184da7b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302742.364393:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302742.364473:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350330624 00000100:00100000:3.0:1789302742.364482:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350330624 00000100:00100000:3.0:1789302742.364566:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 193 00000100:00100000:3.0:1789302742.364578:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26514:x1876219350330624:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1789302742.364708:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26514:x1876219350330624:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 133us (323us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302742.364715:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 193 00000100:00100000:1.0:1789302742.369827:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302742.369901:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350330880 00000100:00100000:3.0:1789302742.369912:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350330880 00000100:00100000:3.0:1789302742.369971:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 194 00000100:00100000:3.0:1789302742.369982:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:26514:x1876219350330880:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302742.370208:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26514:x1876219350330880:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 228us (389us total) trans 4294967924 rc 0/0 00000100:00100000:3.0:1789302742.370216:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 194 00000100:00100000:3.0:1789302742.373621:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.373709:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350331136 00000100:00100000:0.0:1789302742.373720:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350331136 00000100:00100000:0.0:1789302742.373776:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 974 00000100:00100000:0.0:1789302742.373785:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26514:x1876219350331136:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302742.373890:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26514:x1876219350331136:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 106us (277us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302742.373909:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 974 00000100:00100000:3.0:1789302742.376539:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.376935:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350331392 00000100:00100000:1.0:1789302742.376944:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350331392 00000100:00100000:1.0:1789302742.377005:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 975 00000100:00100000:1.0:1789302742.377015:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26514:x1876219350331392:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302742.377035:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302742.377057:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184da89 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04c3 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.377149:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.377164:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.377173:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.377205:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.377218:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 47 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.377231:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371115008:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302742.377311:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58af00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302742.391214:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302742.394661:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350331520 00000100:00100000:3.0:1789302742.394679:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350331520 00010000:00010000:3.0:1789302742.394729:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 48 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302742.394778:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 295 00000100:00100000:3.0:1789302742.394787:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350331520:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302742.394818:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302742.394826:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 48 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.394840:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 48 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.394861:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302742.394867:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.394876:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.394884:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.394902:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302742.394957:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350331520:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 172us (3753us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302742.394965:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 295 00010000:00010000:0.0:1789302742.407071:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184da82 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04bc expref: 47 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302742.407092:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371115008:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302742.407128:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407140:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407145:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302742.407166:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 0 00000004:00080000:0.0:1789302742.407472:0:18427:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x14:0x0], index 1: 0 00010000:00010000:0.0:1789302742.407808:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407818:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407824:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407830:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302742.407834:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407841:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407869:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302742.407872:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302742.407877:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.407882:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184da90 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.408192:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184da89 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04c3 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.408203:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302742.408206:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302742.408209:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184da89 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04c3 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.408220:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7800, rc 0) 00000100:00100000:0.0:1789302742.408322:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26514:x1876219350331392:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 31308us (31826us total) trans 4294967925 rc 301/301 00000100:00100000:0.0:1789302742.408330:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 975 00000100:00100000:0.0:1789302742.427811:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.427883:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350331776 00000100:00100000:2.0:1789302742.427892:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350331776 00000100:00100000:2.0:1789302742.427952:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 195 00000100:00100000:2.0:1789302742.427961:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ee450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:26514:x1876219350331776:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302742.428294:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ee450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:26514:x1876219350331776:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 332us (487us total) trans 4294967926 rc 0/0 00000100:00100000:2.0:1789302742.428302:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 195 00000100:00100000:3.0:1789302742.432690:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.433649:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350331904 00000100:00100000:0.0:1789302742.433660:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350331904 00000100:00100000:0.0:1789302742.433719:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 976 00000100:00100000:0.0:1789302742.433730:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:26514:x1876219350331904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302742.433752:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302742.433772:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184da9e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04ca expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.433850:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184daa5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.433866:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184daa5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.433878:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184daa5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302742.433913:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1a:0x0]: returning size 0 00000004:00200000:0.0:1789302742.433918:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302742.433936:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184daa5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.433963:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184da9e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ca expref: 49 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.433980:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184daa5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ca expref: 48 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.433987:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302742.433990:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1789302742.433993:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516200, rc 0) 00000100:00100000:0.0:1789302742.434059:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:26514:x1876219350331904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 330us (1378us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302742.434066:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 976 00000100:00100000:0.0:1789302742.445447:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.445556:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350332032 00000100:00100000:2.0:1789302742.445566:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350332032 00000100:00100000:2.0:1789302742.445631:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:2.0:1789302742.445641:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350332032:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302742.445658:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302742.445679:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04d1 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302742.445705:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x14:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302742.445724:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=579s v=(0 0) 00010000:00010000:2.0:1789302742.445727:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302742.445736:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.445753:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.445778:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302742.445829:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350332032:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 190us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302742.445836:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00080000:3.0:1789302742.446357:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302742.453194:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.453312:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350332288 00000100:00100000:0.0:1789302742.453329:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350332288 00000100:00100000:0.0:1789302742.453382:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 977 00000100:00100000:0.0:1789302742.453392:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:26514:x1876219350332288:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302742.453492:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:26514:x1876219350332288:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 100us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302742.453520:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 977 00000100:00100000:3.0:1789302742.457846:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.458096:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350332416 00000100:00100000:1.0:1789302742.458107:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350332416 00000100:00100000:1.0:1789302742.458169:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:1.0:1789302742.458180:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:26514:x1876219350332416:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302742.458197:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302742.458216:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04d8 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.458245:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.458254:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302742.458258:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302742.458262:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.458293:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.458304:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 11 pid: 18578 timeout: 679 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.458317:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe33100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371115136:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302742.458395:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58ae40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302742.468938:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe33100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371115136:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302742.468991:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.469013:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:1.0:1789302742.471703:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:26514:x1876219350332416:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 13513us (13853us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302742.471714:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:2.0:1789302742.473388:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.477225:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350332544 00000100:00100000:1.0:1789302742.477239:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350332544 00010000:00010000:1.0:1789302742.477283:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 11 pid: 18578 timeout: 679 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.477324:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 296 00000100:00100000:1.0:1789302742.477335:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350332544:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302742.477350:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302742.477371:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 11 pid: 18578 timeout: 679 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.477385:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 11 pid: 18578 timeout: 679 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.477408:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184daac lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d1 expref: 11 pid: 18578 timeout: 679 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.477423:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302742.477438:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302742.477446:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.477467:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.477523:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.477547:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe33100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371115264:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302742.477626:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58ae40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302742.482917:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe33100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371115264:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302742.482943:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302742.483006:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350332544:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 5672us (9623us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302742.483016:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 296 00000100:00100000:1.0:1789302742.490688:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.491584:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350332672 00000100:00100000:1.0:1789302742.491594:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350332672 00002000:00100000:1.0:1789302742.491643:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x14:0x0] (0->157) req@ffff960f3fe33b80 x1876219350332672/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:318/0 lens 648/0 e 0 to 0 dl 1789302753 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302742.491654:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fe33b80 with x1876219350332672 ext(0->157) 00002000:00010000:1.0:1789302742.491659:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fe33b80 00000100:00100000:1.0:1789302742.491686:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 93 00000100:00100000:1.0:1789302742.491695:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350332672:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:1.0:1789302742.492181:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350332672:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 486us (1501us total) trans 4294967351 rc 0/0 00000100:00100000:1.0:1789302742.492189:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 93 00002000:00100000:1.0:1789302742.492196:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x14:0x0] (0->157) req@ffff960f3fe33b80 x1876219350332672/t4294967351(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:318/0 lens 648/448 e 0 to 0 dl 1789302753 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302742.492210:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fe33b80 with x1876219350332672 ext(0->157) 00002000:00010000:1.0:1789302742.492214:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fe33b80 00000100:00100000:1.0:1789302742.495981:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.496064:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350332800 00000100:00100000:0.0:1789302742.496074:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350332800 00000100:00100000:0.0:1789302742.496130:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 978 00000100:00100000:0.0:1789302742.496141:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:26514:x1876219350332800:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:0.0:1789302742.554730:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967925 is committed 00000001:00080000:0.0:1789302742.554747:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967920 is committed 00000004:00080000:0.0:1789302742.554751:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 12 committed 00000020:00080000:0.0:1789302742.554768:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f258cfa80 refcount 2 committed 0 result 0 batchid 15 00000020:00080000:0.0:1789302742.554773:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960e02228500 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f258cf6c0 00000020:00080000:0.0:1789302742.554776:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960e02228f00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3fa16000 00000020:00080000:2.0:1789302742.554804:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 15 committed batchid 14 00000100:00100000:0.0:1789302742.554896:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371115392:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789302742.554957:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000020:00080000:2.0:1789302742.554969:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 14 00000100:00100000:0.0:1789302742.555028:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371115392 00000100:00100000:0.0:1789302742.555034:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371115392 00000100:00100000:0.0:1789302742.555102:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1789302742.557688:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371115392:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789302742.557723:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:20 count 1 00000001:00080000:2.0:1789302742.557912:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967926 is committed 00002000:00010000:0.0:1789302742.558056:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x14:0x0] initial LVB size: 158, mtime: 0x6aa697d3, atime: 0x0, ctime: 0x6aa697d3, blocks: 0xb 00010000:00010000:0.0:1789302742.558066:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 3/0,1 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.558081:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=579s v=(0 0) 00010000:00010000:0.0:1789302742.558085:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302742.558091:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.558305:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.558318:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.558326:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.560653:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1789302742.561196:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371115392:12345-0@lo:6:osp-syn-0-0.0 Request processed in 3523us (6235us total) trans 4294967347 rc 0/0 00000100:00100000:1.0:1789302742.561206:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00010000:00010000:1.0:1789302742.561240:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302742.561248:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.561259:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302742.561264:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.561274:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.561291:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 1/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.561298:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b03da6c/0x64fe43095184daba lrc: 0/0,0 mode: --/PW res: [0x240000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.561318:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302742.561327:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302742.561329:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302742.561561:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26514:x1876219350332800:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 65421us (65588us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302742.561570:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 978 00000004:00080000:1.0:1789302742.565291:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ffa7b80/2, rc 0, transno 4294967347 00000100:00100000:1.0:1789302742.565306:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371115392:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789302742.568298:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.568332:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350332928 00000100:00100000:2.0:1789302742.568352:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350332928 00000100:00100000:2.0:1789302742.568412:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 256 00000100:00100000:2.0:1789302742.568423:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350332928:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1789302742.605986:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350332928:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 37555us (37687us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302742.605999:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 256 00000001:00080000:2.0:1789302742.606036:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967351 is committed 00000100:00100000:1.0:1789302742.610226:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.610300:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350333184 00000100:00100000:0.0:1789302742.610310:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350333184 00000100:00100000:0.0:1789302742.610366:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 979 00000100:00100000:0.0:1789302742.610376:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26514:x1876219350333184:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302742.610425:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302742.610450:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184daa5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ca expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.610496:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302742.610552:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302742.610581:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.610594:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.610603:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302742.610959:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005757643c reply state 00000000ceeff9ab transno 4294967927 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302742.610969:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302742.611102:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26514:x1876219350333184:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 724us (882us total) trans 4294967927 rc 0/0 00000100:00100000:0.0:1789302742.611196:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 979 00010000:00010000:3.0:1789302742.613474:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.613489:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.613545:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.613551:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302742.613555:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.613563:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.613578:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302742.613581:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302742.613695:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.613704:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dac1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302742.614213:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967345 is committed 00000001:00080000:2.0:1789302742.614227:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967346 is committed 00000100:00100000:0.0:1789302742.614671:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.614734:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350333312 00000100:00100000:1.0:1789302742.614743:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350333312 00000100:00100000:1.0:1789302742.614798:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 980 00000100:00100000:1.0:1789302742.614808:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26514:x1876219350333312:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302742.614832:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302742.614848:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04df expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.614900:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.614914:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.614933:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:1.0:1789302742.614936:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302742.614939:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6e00, rc 0) 00000100:00100000:1.0:1789302742.614994:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26514:x1876219350333312:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 187us (326us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302742.615014:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 980 00000100:00100000:2.0:1789302742.620601:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302742.620725:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350333440 00000100:00100000:3.0:1789302742.620735:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350333440 00000100:00100000:3.0:1789302742.620799:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 257 00000100:00100000:3.0:1789302742.620809:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350333440:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1789302742.621066:0:8571:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1789302742.621071:0:8571:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb mtime from disk: 0 -> 1789302739 00002000:00010000:3.0:1789302742.621073:0:8571:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb atime from disk: 0 -> 1789302739 00002000:00010000:3.0:1789302742.621075:0:8571:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb ctime from disk: 0 -> 1789302739 00002000:00010000:3.0:1789302742.621078:0:8571:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x14:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1789302742.621145:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350333440:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 336us (551us total) trans 4294967352 rc 0/0 00000100:00100000:3.0:1789302742.621163:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 257 00000100:00100000:3.0:1789302742.626780:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.627090:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350333696 00000100:00100000:1.0:1789302742.627101:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350333696 00000100:00100000:1.0:1789302742.627160:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 981 00000100:00100000:1.0:1789302742.627171:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:26514:x1876219350333696:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1789302742.627235:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.627253:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.627262:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.627296:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.627310:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 43 pid: 5845 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.627324:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe32300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371115520:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302742.627413:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2d80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302742.630195:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe32300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371115520:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302742.630226:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302742.633912:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302742.635177:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350333824 00000100:00100000:2.0:1789302742.635188:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350333824 00010000:00010000:2.0:1789302742.635228:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 44 pid: 5845 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302742.635293:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 297 00000100:00100000:2.0:1789302742.635304:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350333824:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302742.635321:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302742.635327:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 44 pid: 5845 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.635339:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 44 pid: 5845 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.635359:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184da74 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04b5 expref: 44 pid: 5845 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.635371:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:2.0:1789302742.635376:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.635385:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.635393:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302742.635427:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302742.635527:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2347:x1876219350333824:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 196us (1595us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302742.635535:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 297 00010000:00010000:0.0:1789302742.641572:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641591:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641627:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641635:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641666:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641674:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302742.641680:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641689:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641708:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302742.641712:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1789302742.641717:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.641724:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dacf lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302742.641838:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26514:x1876219350333696:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 14665us (15063us total) trans 0 rc -22/-22 00000100:00100000:0.0:1789302742.641847:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 981 00000100:00100000:3.0:1789302742.647450:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.647839:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350333952 00000100:00100000:1.0:1789302742.647862:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350333952 00000100:00100000:1.0:1789302742.647927:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 258 00000100:00100000:1.0:1789302742.647937:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350333952:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302742.647982:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.647998:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.648005:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302742.648008:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302742.648011:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.648041:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.648051:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 11 pid: 13439 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.648063:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe32300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371115648:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302742.648137:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2d80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302742.653265:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.653394:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe32300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371115648:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302742.653420:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302742.653555:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350334080 00000100:00100000:3.0:1789302742.653563:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350334080 00010000:00010000:3.0:1789302742.653673:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 12 pid: 13439 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302742.653749:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 298 00000100:00100000:3.0:1789302742.653758:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350334080:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302742.653773:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302742.653797:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 12 pid: 13439 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.653809:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 12 pid: 13439 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.653841:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000009b7703d8/0x64fe43095184dab3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x14:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04d8 expref: 12 pid: 13439 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.653858:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302742.653863:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302742.653878:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.653910:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302742.653939:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302742.653998:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350334080:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 240us (743us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302742.654005:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 298 00010000:00010000:0.0:1789302742.654058:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654079:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654103:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654112:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654121:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654127:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302742.654131:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654140:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654153:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.654160:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe43095184dad6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302742.654216:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350333952:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 6281us (6776us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302742.654224:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 258 00000100:00100000:2.0:1789302742.660704:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302742.661256:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350334336 00000100:00100000:0.0:1789302742.661268:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350334336 00000100:00100000:0.0:1789302742.661318:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 982 00000100:00100000:0.0:1789302742.661328:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:26514:x1876219350334336:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302742.661409:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 3/0,1 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.661427:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.661980:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.662266:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.662280:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.662288:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.662322:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.662362:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 43 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302742.662376:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371115776:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302742.662550:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f142aa180 going to sleep for 11 seconds 00010000:00010000:2.0:1789302742.668826:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302742.669744:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.669808:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350334464 00000100:00100000:1.0:1789302742.669819:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350334464 00010000:00010000:1.0:1789302742.669859:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 44 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302742.669920:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 299 00000100:00100000:1.0:1789302742.669930:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2348:x1876219350334464:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302742.669944:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302742.669951:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 44 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.669964:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 44 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.669985:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:1.0:1789302742.669991:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.670001:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.670009:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.670029:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302742.670082:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b3050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2348:x1876219350334464:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 153us (343us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302742.670090:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 299 00010000:00010000:0.0:1789302742.670252:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dac8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04df expref: 43 pid: 5843 timeout: 679 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302742.670269:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371115776:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302742.670287:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.670297:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.670302:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302742.670485:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967928 00000001:00080000:0.0:1789302742.670788:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967929 00000001:00080000:0.0:1789302742.670838:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967930 00000100:00100000:0.0:1789302742.670920:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371115904:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302742.670968:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302742.671320:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1035b780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302742.674855:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371115904 00000100:00100000:3.0:1789302742.674869:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371115904 00000100:00100000:2.0:1789302742.680297:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 35 00000100:00100000:2.0:1789302742.680312:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371115904:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789302742.680471:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371115904:12345-0@lo:1000:mdt00_001.0 Request processed in 162us (9501us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302742.680478:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 35 00000100:00100000:1.0:1789302742.680744:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371115904:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1789302742.681146:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967931 00000100:00100000:1.0:1789302742.681228:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371116032:0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1789302742.681279:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302742.681317:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2a80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302742.681536:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116032 00000100:00100000:0.0:1789302742.681543:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116032 00000100:00100000:0.0:1789302742.681579:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 36 00000100:00100000:0.0:1789302742.681594:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371116032:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302742.681766:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5844:x1876219371116032:12345-0@lo:1000:mdt00_001.0 Request processed in 173us (486us total) trans 4294967353 rc 0/0 00000100:00100000:0.0:1789302742.681773:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 36 00000100:00100000:1.0:1789302742.681986:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31180 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371116032:0@lo:1000:mdt00_001.0 00000001:00080000:1.0:1789302742.682065:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967932 00010000:00010000:1.0:1789302742.682095:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682116:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682125:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682141:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302742.682145:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682154:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682169:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f03927a00) 00010000:00010000:1.0:1789302742.682172:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302742.682177:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682184:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dae4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682215:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682223:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682236:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682241:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302742.682245:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682252:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682260:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2100) 00010000:00010000:1.0:1789302742.682263:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302742.682266:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682271:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184da19 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682292:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682305:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682314:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682322:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682328:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682341:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682347:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302742.682350:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682356:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682363:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2100) 00010000:00010000:1.0:1789302742.682365:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302742.682367:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682373:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daeb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x19:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.682382:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.684211:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.684225:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.684232:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302742.684236:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.684265:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.684282:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f0cdf2100) 00010000:00010000:1.0:1789302742.684285:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302742.684289:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 2/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.684294:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dadd lrc: 0/0,0 mode: --/PW res: [0x200000402:0x19:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302742.685041:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302742.685199:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:26514:x1876219350334336:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 23867us (24498us total) trans 4294967933 rc 0/0 00000100:00100000:1.0:1789302742.685209:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 982 00000020:00080000:1.0:1789302742.685215:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:0.0:1789302742.686775:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302742.686782:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302742.687320:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1789302742.687330:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302742.691039:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302742.691130:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350334720 00000100:00100000:1.0:1789302742.691141:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350334720 00000100:00100000:1.0:1789302742.691199:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 196 00000100:00100000:1.0:1789302742.691209:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:26514:x1876219350334720:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1789302742.692022:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1a:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1789302742.692133:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967934 00010000:00010000:1.0:1789302742.692179:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.692196:0:12976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.692207:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.692228:0:12976:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.692235:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.692241:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302742.692256:0:12976:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789302742.692436:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302742.692443:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.692459:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302742.692464:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.692472:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.692484:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1a:0x0].0x0 (ffff960f10d24000) 00010000:00010000:0.0:1789302742.692487:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302742.692491:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.692496:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184daf2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1a:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302742.692547:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302742.692549:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302742.692551:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302742.693216:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:26514:x1876219350334720:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 2004us (2179us total) trans 4294967935 rc 0/0 00000100:00100000:1.0:1789302742.693227:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 196 00000020:00080000:1.0:1789302742.693233:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1789302743.692809:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302744.113858:0:19377:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302744.113872:0:19377:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302744.114105:0:19377:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 26 count 4 00000040:00080000:2.0:1789302744.114114:0:19377:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302744.186358:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302744.186797:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350334976 00000100:00100000:0.0:1789302744.186812:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350334976 00000100:00100000:0.0:1789302744.186898:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 983 00000100:00100000:0.0:1789302744.186912:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+86:26526:x1876219350334976:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302744.186949:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302744.186977:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184daf9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04e6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187070:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187087:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187101:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187116:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187123:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187128:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789302744.187313:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302744.187475:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184db0e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187484:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184db0e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187530:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184db0e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302744.187544:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:0.0:1789302744.187661:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187669:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187676:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187682:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302744.187687:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187695:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187708:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x0 (ffff960f03927500) 00010000:00010000:0.0:1789302744.187711:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302744.187718:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187725:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184db00 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187736:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187751:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187757:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187762:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302744.187765:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187771:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187779:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3c:0x0].0x6de40d64 (ffff960f03501500) 00010000:00010000:0.0:1789302744.187781:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302744.187785:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187790:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184db07 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3c:0x0].0x6de40d64 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187841:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184daf9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04e6 expref: 89 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187858:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517800/0x64fe43095184db0e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04e6 expref: 88 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.187866:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f03501300) 00010000:00010000:0.0:1789302744.187868:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302744.187877:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517800, rc 0) 00000100:00100000:0.0:1789302744.187958:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:26526:x1876219350334976:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1049us (1612us total) trans 4294967936 rc 0/0 00000100:00100000:0.0:1789302744.187967:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 983 00000100:00100000:2.0:1789302744.208661:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302744.209537:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350335104 00000100:00100000:0.0:1789302744.209553:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350335104 00000100:00100000:0.0:1789302744.209643:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 984 00000100:00100000:0.0:1789302744.209656:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:26526:x1876219350335104:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:0.0:1789302744.210080:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210104:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210120:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210160:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210169:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210176:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210183:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302744.210188:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210197:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210224:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f03501300) 00010000:00010000:0.0:1789302744.210228:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302744.210234:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.210241:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516c00/0x64fe43095184db1c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302744.210343:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:26526:x1876219350335104:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 688us (1692us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302744.210352:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 984 00000100:00100000:2.0:1789302744.213234:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302744.213269:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350335360 00000100:00100000:2.0:1789302744.213279:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350335360 00000100:00100000:2.0:1789302744.213328:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 197 00000100:00100000:2.0:1789302744.213339:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:26526:x1876219350335360:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:2.0:1789302744.213698:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:26526:x1876219350335360:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 359us (473us total) trans 4294967937 rc 0/0 00000100:00100000:2.0:1789302744.213706:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 197 00000100:00100000:3.0:1789302744.315308:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302744.315434:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350335488 00000100:00100000:0.0:1789302744.315458:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350335488 00000100:00100000:0.0:1789302744.315589:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00080000:1.0:1789302744.315814:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302744.316535:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2346:x1876219350335488:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:0.0:1789302744.316567:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302744.316599:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b707599/0x64fe43095184db23 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04ed expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302744.316640:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x14:0x0] initial LVB size: 158, mtime: 0x6aa697d3, atime: 0x6aa697d3, ctime: 0x6aa697d3, blocks: 0xb 00010000:00010000:0.0:1789302744.316667:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=581s v=(0 0) 00010000:00010000:0.0:1789302744.316672:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x14:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302744.316684:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b707599/0x64fe43095184db23 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ed expref: 14 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.316706:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006b707599/0x64fe43095184db23 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x14:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04ed expref: 14 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302744.316744:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516c00, rc 0) 00000100:00100000:0.0:1789302744.316822:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2346:x1876219350335488:12345-192.168.203.11@tcp:101:diff.0 Request processed in 1222us (1528us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302744.316831:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00100000:2.0:1789302744.320585:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302744.320841:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350335744 00000100:00100000:1.0:1789302744.320855:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350335744 00000100:00100000:1.0:1789302744.320947:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 985 00000100:00100000:1.0:1789302744.320960:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:26531:x1876219350335744:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:1.0:1789302744.320994:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302744.321021:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184db2a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04f4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302744.321178:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184db31 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302744.321197:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184db31 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302744.321220:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184db31 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302744.321235:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x19:0x0]: returning size 158 00010000:00010000:1.0:1789302744.322278:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184db2a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3c:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04f4 expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302744.322303:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184db31 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04f4 expref: 91 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302744.322314:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f03501300) 00010000:00010000:1.0:1789302744.322317:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302744.322322:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6800, rc 0) 00000100:00100000:1.0:1789302744.323214:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:26531:x1876219350335744:12345-192.168.203.11@tcp:101:diff.0 Request processed in 2249us (2636us total) trans 4294967938 rc 0/0 00000100:00100000:1.0:1789302744.323227:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 985 00000100:00100000:2.0:1789302744.327885:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302744.327964:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302744.329387:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350335872 00000100:00100000:0.0:1789302744.329406:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350335872 00002000:00100000:0.0:1789302744.329486:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x14:0x0] (0->4095) req@ffff960e02323100 x1876219350335872/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:320/0 lens 488/0 e 0 to 0 dl 1789302755 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302744.329538:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02323100 with x1876219350335872 ext(0->4095) 00002000:00010000:0.0:1789302744.329546:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e02323100 00000100:00100000:0.0:1789302744.329584:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 94 00000100:00100000:0.0:1789302744.329595:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350335872:12345-192.168.203.11@tcp:3:diff.0 00000100:00100000:0.0:1789302744.329892:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350335872:12345-192.168.203.11@tcp:3:diff.0 Request processed in 297us (2024us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302744.329900:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 94 00002000:00100000:0.0:1789302744.329906:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x14:0x0] (0->4095) req@ffff960e02323100 x1876219350335872/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:320/0 lens 488/4536 e 0 to 0 dl 1789302755 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302744.329914:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02323100 with x1876219350335872 ext(0->4095) 00002000:00010000:0.0:1789302744.329918:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e02323100 00000100:00100000:1.0:1789302744.330012:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350336000 00000100:00100000:1.0:1789302744.330018:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350336000 00000100:00100000:1.0:1789302744.330067:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 43 00000100:00100000:1.0:1789302744.330074:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2348:x1876219350336000:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302744.330086:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302744.330117:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184db31 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x19:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04f4 expref: 91 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302744.330136:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x19:0x0].0x0 (ffff960f03501300) 00010000:00010000:1.0:1789302744.330141:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302744.330182:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:2348:x1876219350336000:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 109us (2219us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302744.330189:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 43 00000100:00100000:2.0:1789302744.335859:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302744.336042:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350336256 00000100:00100000:1.0:1789302744.336056:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350336256 00000100:00100000:1.0:1789302744.336132:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 198 00000100:00100000:1.0:1789302744.336145:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:26531:x1876219350336256:12345-192.168.203.11@tcp:35:diff.0 00000100:00100000:1.0:1789302744.336692:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:26531:x1876219350336256:12345-192.168.203.11@tcp:35:diff.0 Request processed in 548us (845us total) trans 4294967939 rc 0/0 00000100:00100000:1.0:1789302744.336701:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 198 00010000:00010000:2.0:1789302744.717310:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302745.102379:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371116160:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.102534:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302745.102595:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371116672:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.102638:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302745.102669:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789302745.102675:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371116288:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.102698:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302745.102718:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371116800:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.102744:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302745.102780:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116672 00000100:00100000:2.0:1789302745.102790:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116672 00000100:00100000:2.0:1789302745.102866:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116288 00000100:00100000:2.0:1789302745.102870:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116288 00000100:00100000:2.0:1789302745.102895:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1789302745.102906:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371116672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.102973:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371116672:12345-0@lo:400:kworker.0 Request processed in 69us (333us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.102980:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 375 00000100:00100000:2.0:1789302745.102998:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:2.0:1789302745.103004:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371116288:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.103026:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371116288:12345-0@lo:400:kworker.0 Request processed in 22us (327us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.103032:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00080000:2.0:1789302745.103052:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302745.103082:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371116288:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.103092:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371116672:0@lo:400:kworker.0 00000100:00100000:1.0:1789302745.103540:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371116416:0@lo:400:kworker.0 00000100:00100000:1.0:1789302745.103582:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302745.103608:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302745.103615:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371116544:0@lo:400:kworker.0 00000100:00100000:1.0:1789302745.103638:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302745.103678:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116800 00000100:00100000:1.0:1789302745.103684:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116800 00000100:00100000:1.0:1789302745.103720:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116544 00000100:00100000:1.0:1789302745.103723:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116544 00000100:00100000:1.0:1789302745.103743:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1789302745.103750:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371116800:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302745.103798:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371116800:12345-0@lo:400:kworker.0 Request processed in 49us (1053us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302745.103805:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:1.0:1789302745.103819:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 365 00000100:00100000:1.0:1789302745.103838:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371116544:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302745.103891:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371116544:12345-0@lo:400:kworker.0 Request processed in 67us (252us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302745.103898:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 365 00000100:00100000:0.0:1789302745.103936:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116160 00000100:00100000:0.0:1789302745.103949:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116160 00000100:00100000:0.0:1789302745.104015:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 552 00000100:00100000:0.0:1789302745.104025:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371116160:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302745.104087:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967346 gen 2 00000004:00080000:1.0:1789302745.104095:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967346 committed transno 4294967346 00000100:00100000:0.0:1789302745.104127:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371116160:12345-0@lo:400:kworker.0 Request processed in 101us (1623us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302745.104132:0:3300:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff960f3ffa7b80 x1876219371115392/t4294967347(4294967347) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789302758 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:0.0:1789302745.104142:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 552 00000100:00100000:1.0:1789302745.104162:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371116544:0@lo:400:kworker.0 00000100:00080000:2.0:1789302745.104256:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302745.104261:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302745.104287:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371116800:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.104299:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371116160:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.104325:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116416 00000100:00100000:2.0:1789302745.104330:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116416 00000100:00100000:2.0:1789302745.104378:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1789302745.104385:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371116416:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789302745.104463:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302745.130103:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371116416:12345-0@lo:400:kworker.0 Request processed in 25702us (26504us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.130123:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:1.0:1789302745.130385:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371116416:0@lo:400:kworker.0 00000100:00100000:2.0:1789302745.379064:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302745.379108:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302745.379134:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302745.379156:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302745.379172:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302745.379191:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302745.379211:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302745.379236:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350336512 00000100:00100000:2.0:1789302745.379244:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350336512 00000100:00100000:2.0:1789302745.379283:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350336896 00000100:00100000:2.0:1789302745.379285:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350336896 00000100:00100000:2.0:1789302745.379309:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 378 00000100:00100000:2.0:1789302745.379316:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2347:x1876219350336512:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302745.379379:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2347:x1876219350336512:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 62us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.379402:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 378 00000100:00100000:2.0:1789302745.379418:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 379 00000100:00100000:2.0:1789302745.379424:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350336896:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302745.379461:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350336896:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (287us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.379466:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 379 00000100:00100000:2.0:1789302745.379485:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350336640 00000100:00100000:2.0:1789302745.379488:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350336640 00000100:00100000:2.0:1789302745.379535:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350336768 00000100:00100000:2.0:1789302745.379538:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350336768 00000100:00100000:2.0:1789302745.379551:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337024 00000100:00100000:2.0:1789302745.379553:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337024 00000100:00100000:2.0:1789302745.379562:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337152 00000100:00100000:2.0:1789302745.379564:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337152 00000100:00100000:2.0:1789302745.379577:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:2.0:1789302745.379582:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350336640:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302745.379603:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350336640:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.379608:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:2.0:1789302745.379618:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 367 00000100:00100000:2.0:1789302745.379623:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2347:x1876219350336768:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302745.379639:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2347:x1876219350336768:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 17us (483us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.379645:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 367 00000100:00100000:2.0:1789302745.379653:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 368 00000100:00100000:2.0:1789302745.379673:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350337024:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302745.379694:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350337024:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (502us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.379699:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 368 00000100:00100000:2.0:1789302745.379709:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 369 00000100:00100000:2.0:1789302745.379714:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350337152:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302745.379729:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350337152:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 15us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302745.379734:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 369 00000100:00080000:2.0:1789302745.379747:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302745.379751:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302745.379753:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302745.380672:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350336384 00000100:00100000:1.0:1789302745.380680:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350336384 00000100:00100000:1.0:1789302745.380719:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 553 00000100:00100000:1.0:1789302745.380726:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350336384:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302745.380779:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350336384:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 54us (1725us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302745.380785:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 553 00000100:00100000:1.0:1789302745.612834:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371116928:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302745.612904:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302745.616375:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371116928 00000100:00100000:0.0:1789302745.616391:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371116928 00000100:00100000:1.0:1789302745.616649:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 162 00000100:00100000:1.0:1789302745.616661:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371116928:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302745.616700:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 74776576 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302745.616849:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371116928:12345-0@lo:13:osp-pre-1-0.0 Request processed in 189us (3943us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302745.616857:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 162 00000100:00100000:2.0:1789302745.619202:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371116928:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302745.743651:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302746.127762:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371117056:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302746.127831:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302746.128456:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117056 00000100:00100000:0.0:1789302746.128467:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117056 00000100:00100000:0.0:1789302746.128560:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:0.0:1789302746.128570:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371117056:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302746.128605:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 74776576 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302746.128661:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371117056:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (830us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302746.128669:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:0.0:1789302746.128713:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371117056:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302746.765879:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789302747.757433:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967347 is committed 00000001:00080000:3.0:1789302747.781704:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967936 is committed 00000001:00080000:3.0:1789302747.781718:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967931 is committed 00000001:00080000:3.0:1789302747.781730:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967939 is committed 00000001:00080000:3.0:1789302747.781735:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967935 is committed 00000020:00080000:3.0:1789302747.781748:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f098a2600 refcount 2 committed 0 result 0 batchid 16 00000020:00080000:3.0:1789302747.781753:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3b0b4b00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f098a2900 00000020:00080000:3.0:1789302747.781756:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3b0b4400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3acf7400 00000020:00080000:3.0:1789302747.781789:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 15 batchid updated 00000020:00080000:0.0:1789302747.781832:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 16 committed batchid 15 00000020:00080000:0.0:1789302747.782014:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 15 00010000:00010000:2.0:1789302747.790784:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302747.801678:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967353 is committed 00000001:00080000:2.0:1789302747.801695:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967352 is committed 00010000:00010000:1.0:1789302748.813271:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302749.840696:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302749.840896:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302749.840902:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302749.841199:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 26 count 3 00000040:00080000:3.0:1789302749.841206:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302750.221632:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371117568:0@lo:400:kworker.0 00000100:00100000:1.0:1789302750.221696:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302750.221754:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302750.221761:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371117440:0@lo:400:kworker.0 00000100:00100000:1.0:1789302750.221787:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302750.221858:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371117312:0@lo:400:kworker.0 00000100:00100000:3.0:1789302750.221901:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302750.221935:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371117824:0@lo:400:kworker.0 00000100:00100000:3.0:1789302750.221955:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302750.221970:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789302750.221975:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371117184:0@lo:400:kworker.0 00000100:00100000:3.0:1789302750.222005:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302750.222022:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371117696:0@lo:400:kworker.0 00000100:00100000:3.0:1789302750.222040:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302750.222193:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117568 00000100:00100000:2.0:1789302750.222202:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117568 00000100:00100000:2.0:1789302750.222259:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117824 00000100:00100000:2.0:1789302750.222263:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117824 00000100:00100000:2.0:1789302750.222286:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789302750.222295:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371117568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.222347:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371117568:12345-0@lo:400:kworker.0 Request processed in 54us (651us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302750.222354:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 370 00000100:00100000:2.0:1789302750.222370:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1789302750.222384:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371117824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.222429:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371117824:12345-0@lo:400:kworker.0 Request processed in 46us (473us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302750.222435:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 371 00000100:00100000:2.0:1789302750.222452:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117440 00000100:00100000:2.0:1789302750.222456:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117440 00000100:00100000:2.0:1789302750.222480:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117312 00000100:00100000:2.0:1789302750.222483:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117312 00000100:00100000:2.0:1789302750.222498:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117696 00000100:00100000:2.0:1789302750.222525:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117696 00000100:00100000:2.0:1789302750.222541:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:2.0:1789302750.222547:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371117440:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.222582:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371117440:12345-0@lo:400:kworker.0 Request processed in 35us (795us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302750.222588:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:2.0:1789302750.222599:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:2.0:1789302750.222604:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371117312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.222629:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371117312:12345-0@lo:400:kworker.0 Request processed in 25us (728us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302750.222634:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00000100:00100000:2.0:1789302750.222644:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1789302750.222649:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371117696:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.222674:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371117696:12345-0@lo:400:kworker.0 Request processed in 25us (633us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302750.222680:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 382 00000100:00100000:2.0:1789302750.222738:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe33480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371117440:0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.222762:0:3301:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967347 gen 2 00000004:00080000:2.0:1789302750.222766:0:3301:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ffa7b80, transno 4294967347 00000100:00100000:2.0:1789302750.222777:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe33800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371117568:0@lo:400:kworker.0 00000100:00100000:0.0:1789302750.223187:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117184 00000100:00100000:0.0:1789302750.223192:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117184 00000100:00100000:0.0:1789302750.223223:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 554 00000100:00100000:0.0:1789302750.223229:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371117184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302750.223266:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371117184:12345-0@lo:400:kworker.0 Request processed in 37us (1259us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302750.223282:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 554 00000100:00080000:3.0:1789302750.223602:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302750.223607:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302750.223609:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302750.223750:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371117184:0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.223762:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371117696:0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.223770:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371117312:0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.223779:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967353 gen 2 00000004:00080000:2.0:1789302750.223783:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967353 committed transno 4294967353 00000100:00100000:2.0:1789302750.223794:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371117824:0@lo:400:kworker.0 00000100:00100000:2.0:1789302750.464811:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302750.464948:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337280 00000100:00100000:0.0:1789302750.464958:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337280 00000100:00080000:3.0:1789302750.465040:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302750.465047:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 986 00000100:00100000:0.0:1789302750.465061:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26485:x1876219350337280:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1789302750.465165:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:26485:x1876219350337280:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 108us (364us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302750.465173:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 986 00000100:00100000:3.0:1789302750.489674:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302750.489721:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302750.489749:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302750.489792:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302750.489808:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302750.489855:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302750.489947:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337792 00000100:00100000:1.0:1789302750.489957:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337792 00000100:00100000:1.0:1789302750.490017:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350338048 00000100:00100000:1.0:1789302750.490021:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350338048 00000100:00100000:1.0:1789302750.490039:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337664 00000100:00100000:1.0:1789302750.490042:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337664 00000100:00100000:1.0:1789302750.490054:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337920 00000100:00100000:1.0:1789302750.490056:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337920 00000100:00100000:1.0:1789302750.490071:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 372 00000100:00100000:1.0:1789302750.490079:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350337792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302750.490143:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350337792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 65us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302750.490151:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 372 00000100:00100000:1.0:1789302750.490167:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 373 00000100:00100000:1.0:1789302750.490173:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350338048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302750.490196:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350338048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (446us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302750.490201:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 373 00000100:00100000:1.0:1789302750.490212:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 374 00000100:00100000:1.0:1789302750.490229:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350337664:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302750.490254:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350337664:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302750.490271:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 374 00000100:00100000:1.0:1789302750.490283:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 375 00000100:00100000:1.0:1789302750.490289:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350337920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302750.490310:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350337920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (456us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302750.490316:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 375 00000100:00100000:2.0:1789302750.490526:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337536 00000100:00100000:2.0:1789302750.490531:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337536 00000100:00100000:2.0:1789302750.490572:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00100000:2.0:1789302750.490580:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:2348:x1876219350337536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302750.490617:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:2348:x1876219350337536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (954us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302750.490623:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00080000:3.0:1789302750.490751:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302750.490756:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302750.490758:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302750.490822:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350337408 00000100:00100000:0.0:1789302750.490827:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350337408 00000100:00100000:0.0:1789302750.490864:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 555 00000100:00100000:0.0:1789302750.490872:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350337408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302750.490910:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350337408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 39us (1116us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302750.490916:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 555 00000100:00100000:2.0:1789302750.732912:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371117952:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302750.732981:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302750.733106:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371117952 00000100:00100000:0.0:1789302750.733115:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371117952 00000100:00100000:0.0:1789302750.733165:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 164 00000100:00100000:0.0:1789302750.733174:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371117952:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302750.733208:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 74776576 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302750.733259:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371117952:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302750.733267:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 164 00000100:00100000:0.0:1789302750.733310:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371117952:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302750.862941:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302751.244831:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302751.244869:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371118080:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302751.244934:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302751.245058:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371118080 00000100:00100000:2.0:1789302751.245067:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371118080 00000100:00100000:2.0:1789302751.245116:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 165 00000100:00100000:2.0:1789302751.245125:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371118080:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302751.245164:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 74776576 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302751.245217:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371118080:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (282us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302751.245234:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 165 00000100:00100000:0.0:1789302751.245387:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371118080:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302751.886984:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302752.622744:0:19577:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12g: Restore a released file implicitly ========================================================== 08:32:30 (1789302750) 00000020:00080000:3.0:1789302752.859820:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 16 batchid updated 00000020:00080000:3.0:1789302752.859850:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 16 00000020:00080000:3.0:1789302752.859858:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 16 00010000:00010000:0.0:1789302752.910436:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302753.178971:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.179094:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350338176 00000100:00100000:2.0:1789302753.179104:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350338176 00000100:00100000:2.0:1789302753.179184:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 987 00000100:00100000:2.0:1789302753.179196:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:27134:x1876219350338176:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302753.179309:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:27134:x1876219350338176:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 116us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302753.179316:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 987 00000100:00080000:3.0:1789302753.180183:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302753.219907:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.220012:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350338304 00000100:00100000:2.0:1789302753.220022:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350338304 00000100:00100000:2.0:1789302753.220101:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 988 00000100:00100000:2.0:1789302753.220112:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:27135:x1876219350338304:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302753.220131:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302753.220152:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db3f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a04fb expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220231:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220247:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220260:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220297:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220305:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220312:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220596:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220605:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220612:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220618:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.220622:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220629:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220654:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302753.220657:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.220661:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220667:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db46 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220687:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.220693:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222107:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222121:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.222126:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222135:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222149:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff960f03927700) 00010000:00010000:2.0:1789302753.222152:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.222156:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222162:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db4d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222221:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db3f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04fb expref: 88 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222232:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302753.222234:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.222252:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db3f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a04fb expref: 88 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.222261:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3800, rc 0) 00000100:00100000:2.0:1789302753.222333:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:27135:x1876219350338304:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2223us (2436us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302753.222342:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 988 00000100:00080000:3.0:1789302753.223230:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302753.229597:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.229929:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350338560 00000100:00100000:2.0:1789302753.229940:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350338560 00000100:00100000:2.0:1789302753.229992:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 989 00000100:00100000:2.0:1789302753.230003:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:27135:x1876219350338560:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:2.0:1789302753.230062:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302753.230088:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184d875 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a040d expref: 87 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.230108:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302753.230113:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302753.230330:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.230345:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 39 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.230354:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.230395:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 39 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.230423:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 39 pid: 5843 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302753.230437:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c46680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371118208:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302753.230570:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302753.247431:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.248601:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c46680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371118208:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302753.248634:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302753.250363:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350338688 00000100:00100000:0.0:1789302753.250376:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350338688 00010000:00010000:0.0:1789302753.250438:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 40 pid: 5843 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302753.250539:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:0.0:1789302753.250551:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:2347:x1876219350338688:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302753.250566:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302753.250572:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 40 pid: 5843 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.250586:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 40 pid: 5843 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.250608:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184d8b4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0422 expref: 40 pid: 5843 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.250624:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:0.0:1789302753.250628:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.250646:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.250671:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.250983:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302753.251037:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:2347:x1876219350338688:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 487us (3614us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.251044:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 300 00010000:00010000:2.0:1789302753.251127:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.251136:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.251160:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.251180:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.251189:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302753.252264:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 16384 00010000:00010000:2.0:1789302753.252276:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.252285:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.252298:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.252304:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.252308:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.252317:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.252328:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302753.252330:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.252335:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.252341:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db54 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302753.252357:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000bde6eea reply state 00000000aa5afc5b transno 4294967940 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302753.252365:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302753.252442:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:27135:x1876219350338560:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 22439us (22854us total) trans 4294967940 rc 0/0 00000100:00100000:2.0:1789302753.252451:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 989 00010000:00010000:0.0:1789302753.257240:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.257260:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.257268:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.257274:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302753.257279:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.257288:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.257304:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff960f03927700) 00010000:00010000:0.0:1789302753.257308:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.257313:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.257319:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184db5b lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302753.289616:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.290547:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350338816 00000100:00100000:0.0:1789302753.290558:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350338816 00000100:00100000:0.0:1789302753.290631:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 990 00000100:00100000:0.0:1789302753.290641:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+87:27136:x1876219350338816:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302753.290664:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.290688:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184db62 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0502 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.291901:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184db69 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.291919:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184db69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.291930:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184db69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302753.291966:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302753.291969:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302753.291989:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184db69 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.292017:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184db62 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0502 expref: 89 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.292039:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184db69 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0502 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.292047:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302753.292065:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302753.292069:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517400, rc 0) 00000100:00100000:0.0:1789302753.292131:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:27136:x1876219350338816:12345-192.168.203.11@tcp:101:bash.0 Request processed in 1490us (2786us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.292138:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 990 00000100:00100000:3.0:1789302753.299026:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.299292:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350338944 00000100:00100000:2.0:1789302753.299305:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350338944 00000100:00100000:2.0:1789302753.299372:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 991 00000100:00100000:2.0:1789302753.299393:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+88:27136:x1876219350338944:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302753.299430:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302753.299941:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db70 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0509 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300046:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300232:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300248:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300266:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300273:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300281:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300853:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3400/0x64fe43095184db85 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300878:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3400/0x64fe43095184db85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.300892:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3400/0x64fe43095184db85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302753.301058:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 16384 00000004:00200000:2.0:1789302753.301078:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302753.301102:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3400/0x64fe43095184db85 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301117:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301123:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301130:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301136:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.301140:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301147:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301158:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302753.301161:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.301166:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301173:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184db77 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301190:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301196:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301212:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301217:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.301219:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301226:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301235:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff960f03927a00) 00010000:00010000:2.0:1789302753.301237:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.301240:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301245:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184db7e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301287:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184db70 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0509 expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301301:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3400/0x64fe43095184db85 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0509 expref: 89 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.301308:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:2.0:1789302753.301310:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302753.301313:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3400, rc 0) 00000100:00100000:2.0:1789302753.301903:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:27136:x1876219350338944:12345-192.168.203.11@tcp:101:bash.0 Request processed in 2508us (2886us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302753.301934:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 991 00000100:00100000:1.0:1789302753.317264:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.326632:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350339072 00000100:00100000:2.0:1789302753.326649:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350339072 00000100:00100000:3.0:1789302753.330245:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 992 00000100:00100000:3.0:1789302753.330261:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:27136:x1876219350339072:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302753.330282:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302753.330303:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184db8c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0510 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330692:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330709:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330722:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330740:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330796:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330802:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330867:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330873:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330880:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330898:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302753.330902:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330909:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330921:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:3.0:1789302753.330924:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302753.330929:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330936:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184db93 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330948:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330954:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330959:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330965:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302753.330967:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330974:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.330992:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f0cdf2a00) 00010000:00010000:3.0:1789302753.330995:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302753.330998:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.331004:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184db9a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.331037:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184db8c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0510 expref: 90 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.331048:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:3.0:1789302753.331051:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302753.331055:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184db8c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0510 expref: 90 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.331064:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:3.0:1789302753.331135:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:27136:x1876219350339072:12345-192.168.203.11@tcp:101:bash.0 Request processed in 878us (13879us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302753.331143:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 992 00000100:00100000:3.0:1789302753.342781:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.342937:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350339200 00000100:00100000:0.0:1789302753.342948:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350339200 00000100:00100000:0.0:1789302753.343001:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 993 00000100:00100000:0.0:1789302753.343011:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+39:27137:x1876219350339200:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302753.343031:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.343053:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dba1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0517 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.343137:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.343151:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.343162:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302753.343199:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302753.343203:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302753.343223:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.343246:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dba1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 41 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.343277:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 40 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.343285:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302753.343289:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302753.343293:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517200, rc 0) 00000100:00100000:0.0:1789302753.343357:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:27137:x1876219350339200:12345-192.168.203.11@tcp:101:rm.0 Request processed in 348us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.343364:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 993 00000100:00100000:0.0:1789302753.348318:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.348416:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350339328 00000100:00100000:2.0:1789302753.348425:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350339328 00000100:00100000:2.0:1789302753.348480:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 994 00000100:00100000:2.0:1789302753.348490:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:27137:x1876219350339328:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302753.348993:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302753.349017:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184dbaf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a051e expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349083:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349097:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349108:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349122:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349128:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349134:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349224:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184dbc4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349248:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184dbc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349255:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184dbc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302753.349284:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 16384 00000004:00200000:2.0:1789302753.349288:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302753.349306:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184dbc4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349318:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349325:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349332:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349338:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.349343:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349350:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349360:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302753.349363:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.349367:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349729:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbb6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349826:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349984:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349990:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.349994:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.349998:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.350005:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.350016:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x8c43700b (ffff960f03927a00) 00010000:00010000:2.0:1789302753.350018:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.350022:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.350027:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbbd lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x8c43700b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.350063:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184dbaf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a051e expref: 42 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.350090:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2c00/0x64fe43095184dbc4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a051e expref: 41 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.350096:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:2.0:1789302753.350099:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302753.350101:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2c00, rc 0) 00000100:00100000:3.0:1789302753.359949:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27137:x1876219350339328:12345-192.168.203.11@tcp:101:rm.0 Request processed in 11449us (11629us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302753.359963:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 994 00000100:00100000:2.0:1789302753.366296:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302753.366390:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350339456 00000100:00100000:3.0:1789302753.366400:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350339456 00000100:00100000:3.0:1789302753.366464:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 995 00000100:00100000:3.0:1789302753.366491:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27137:x1876219350339456:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302753.366556:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302753.366576:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dbcb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0525 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366640:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366654:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366665:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366681:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366687:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366693:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366759:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366767:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366774:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366790:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302753.366796:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366804:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366816:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:3.0:1789302753.366819:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302753.366824:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366843:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dbd2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366857:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366864:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366869:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366874:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302753.366877:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366884:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366901:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f0cdf2a00) 00010000:00010000:3.0:1789302753.366904:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302753.366906:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366912:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbd9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366942:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dbcb lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0525 expref: 42 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366953:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:3.0:1789302753.366955:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302753.366959:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dbcb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0525 expref: 42 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.366968:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:3.0:1789302753.367025:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27137:x1876219350339456:12345-192.168.203.11@tcp:101:rm.0 Request processed in 536us (737us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302753.367033:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 995 00000100:00100000:1.0:1789302753.375042:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.375207:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350339584 00000100:00100000:2.0:1789302753.375217:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350339584 00000100:00100000:2.0:1789302753.375285:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 996 00000100:00100000:2.0:1789302753.375296:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27137:x1876219350339584:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302753.375315:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302753.375335:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184dbe0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a052c expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375418:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375432:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375443:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375458:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375465:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375488:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375581:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375589:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375596:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375602:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.375606:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375615:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375626:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:2.0:1789302753.375629:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.375634:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375640:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184dbe7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375653:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375659:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375671:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375676:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302753.375679:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375684:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375692:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f03927a00) 00010000:00010000:2.0:1789302753.375695:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.375698:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375703:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184dbee lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375735:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184dbe0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a052c expref: 42 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375745:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:2.0:1789302753.375747:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302753.375750:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184dbe0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a052c expref: 42 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.375759:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302753.375807:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27137:x1876219350339584:12345-192.168.203.11@tcp:101:rm.0 Request processed in 514us (775us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302753.375822:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 996 00000100:00100000:1.0:1789302753.402100:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302753.402265:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350339712 00000100:00100000:3.0:1789302753.402279:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350339712 00000100:00100000:3.0:1789302753.402353:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 997 00000100:00100000:3.0:1789302753.402366:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27138:x1876219350339712:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302753.402404:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302753.402444:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dbf5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0533 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.402609:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.402628:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.402644:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.402859:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.402883:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.402896:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.402969:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.402977:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.402983:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.402989:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302753.403006:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403014:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403027:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302753.403030:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.403035:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403041:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dbfc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403056:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403061:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403067:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403071:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302753.403074:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403079:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403086:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f03501600) 00010000:00010000:0.0:1789302753.403089:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.403098:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403104:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc03 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403149:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dbf5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0533 expref: 42 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403159:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302753.403162:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.403166:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dbf5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0533 expref: 42 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.403175:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:0.0:1789302753.403259:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27138:x1876219350339712:12345-192.168.203.11@tcp:101:cp.0 Request processed in 895us (1174us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302753.403268:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 997 00000100:00100000:1.0:1789302753.409201:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.409295:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350339840 00000100:00100000:0.0:1789302753.409306:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350339840 00000100:00100000:0.0:1789302753.409367:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 998 00000100:00100000:0.0:1789302753.409391:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27138:x1876219350339840:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302753.409410:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.409429:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc0a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a053a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409489:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409547:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409559:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409574:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409581:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409586:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409643:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409650:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409655:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409661:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302753.409665:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409672:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409683:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302753.409691:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.409697:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409703:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc11 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409716:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409720:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409726:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409730:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302753.409733:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409738:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409745:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f03501600) 00010000:00010000:0.0:1789302753.409747:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.409750:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409754:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409783:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc0a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a053a expref: 42 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409798:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302753.409800:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.409804:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc0a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a053a expref: 42 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.409812:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302753.410349:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27138:x1876219350339840:12345-192.168.203.11@tcp:101:cp.0 Request processed in 958us (1153us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302753.410357:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 998 00000100:00100000:1.0:1789302753.426550:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.426793:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350340096 00000100:00100000:0.0:1789302753.426805:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350340096 00000100:00100000:0.0:1789302753.426882:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 999 00000100:00100000:0.0:1789302753.426893:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27138:x1876219350340096:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302753.426930:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.426957:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc1f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0541 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.427079:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.427094:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.427104:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.427120:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.427142:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.427149:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302753.429290:0:5845:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x15:0x0], index 0: 0 00010000:00010000:0.0:1789302753.430108:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430126:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430138:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430175:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430182:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430189:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430195:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302753.430199:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430207:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430235:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302753.430238:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302753.430244:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430250:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc26 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302753.430264:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000069bff20b reply state 00000000172c43b4 transno 4294967941 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302753.430272:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789302753.430308:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc1f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 44 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430324:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.430332:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302753.430334:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302753.430337:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:0.0:1789302753.433389:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27138:x1876219350340096:12345-192.168.203.11@tcp:101:cp.0 Request processed in 6470us (6820us total) trans 4294967941 rc 0/0 00000100:00100000:0.0:1789302753.433402:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 999 00010000:00010000:3.0:1789302753.435695:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.435711:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.435719:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.435725:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302753.435730:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.438183:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.438204:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f03501600) 00010000:00010000:3.0:1789302753.438207:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302753.438213:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.438221:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dc2d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302753.439938:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.440069:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350340224 00000100:00100000:0.0:1789302753.440079:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350340224 00000100:00100000:0.0:1789302753.440143:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 260 00000100:00100000:0.0:1789302753.440153:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350340224:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302753.440184:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.440204:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0548 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302753.440228:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302753.440246:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=590s v=(0 0) 00010000:00010000:0.0:1789302753.440250:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302753.440259:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.440279:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.440292:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302753.440341:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350340224:12345-192.168.203.11@tcp:101:cp.0 Request processed in 190us (421us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.440348:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 260 00000100:00080000:3.0:1789302753.440465:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302753.444488:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302753.444619:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350340352 00000100:00100000:3.0:1789302753.444630:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350340352 00000100:00100000:3.0:1789302753.444678:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:3.0:1789302753.444688:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:27138:x1876219350340352:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302753.444703:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302753.444721:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a054f expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.444768:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.444779:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302753.444784:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302753.444790:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.444824:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.444838:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 11 pid: 6696 timeout: 690 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302753.444853:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdfb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371118336:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302753.445056:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302753.452497:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302753.452673:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdfb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371118336:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302753.452707:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.452729:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302753.452794:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:27138:x1876219350340352:12345-192.168.203.11@tcp:101:cp.0 Request processed in 8107us (8314us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302753.452801:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:2.0:1789302753.459408:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350340480 00000100:00100000:2.0:1789302753.459423:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350340480 00010000:00010000:0.0:1789302753.461429:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 11 pid: 6696 timeout: 690 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302753.462228:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:0.0:1789302753.462243:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350340480:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302753.462261:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302753.462285:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 11 pid: 6696 timeout: 690 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.462322:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 11 pid: 6696 timeout: 690 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.462345:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc42 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0548 expref: 11 pid: 6696 timeout: 690 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.462363:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302753.462368:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302753.462385:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.462396:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.462433:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302753.462447:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371118464:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302753.462983:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302753.466223:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371118464:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302753.466250:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302753.466314:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350340480:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4075us (13825us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.466323:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:3.0:1789302753.466658:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302753.467088:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350340736 00000100:00100000:3.0:1789302753.467100:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350340736 00000100:00100000:3.0:1789302753.467170:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 199 00000100:00100000:3.0:1789302753.467181:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:27138:x1876219350340736:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:3.0:1789302753.468946:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27138:x1876219350340736:12345-192.168.203.11@tcp:35:cp.0 Request processed in 1755us (2284us total) trans 4294967942 rc 0/0 00000100:00100000:3.0:1789302753.468970:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 199 00000100:00100000:3.0:1789302753.528673:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302753.528919:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350340864 00000100:00100000:1.0:1789302753.528934:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350340864 00000100:00100000:1.0:1789302753.529027:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1000 00000100:00100000:1.0:1789302753.529039:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:27141:x1876219350340864:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302753.529084:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302753.529113:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184dc50 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0556 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529202:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529221:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529234:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529252:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529260:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529268:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529393:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529403:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529410:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302753.529466:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00000004:00200000:1.0:1789302753.529478:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302753.529545:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529561:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529569:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529576:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529583:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529589:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302753.529593:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529601:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529615:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:1.0:1789302753.529618:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302753.529623:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529630:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184dc57 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529644:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529660:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529667:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529672:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302753.529675:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529681:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529690:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f10d24000) 00010000:00010000:1.0:1789302753.529693:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302753.529696:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529702:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dc5e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529741:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184dc50 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 91 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529757:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 90 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.529764:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:1.0:1789302753.529766:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302753.529776:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7800, rc 0) 00000100:00100000:1.0:1789302753.529858:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:27141:x1876219350340864:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 822us (1202us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302753.529866:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1000 00000100:00100000:2.0:1789302753.539906:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.540013:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350340992 00000100:00100000:0.0:1789302753.540033:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350340992 00000100:00100000:0.0:1789302753.540139:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 262 00000100:00100000:0.0:1789302753.540152:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350340992:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302753.540178:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.540202:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc6c lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a055d expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302753.540294:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371118592:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302753.541063:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf840 going to sleep for 11 seconds 00000100:00080000:3.0:1789302753.541182:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789302753.548105:0:8571:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1789302753.548118:0:8571:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb mtime: 0 -> 1789302753 00002000:00010000:0.0:1789302753.548121:0:8571:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb atime: 0 -> 1789302753 00002000:00010000:0.0:1789302753.548123:0:8571:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x15:0x0] updating lvb ctime: 0 -> 1789302753 00000100:00100000:0.0:1789302753.548157:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371118592:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302753.548207:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc6c lrc: 1/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a055d expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.548237:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184dc6c lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a055d expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.548253:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302753.548977:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350340992:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 8823us (9076us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302753.548988:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 262 00000100:00100000:1.0:1789302753.557792:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.557919:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350341120 00000100:00100000:0.0:1789302753.557959:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350341120 00000100:00100000:0.0:1789302753.558029:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1001 00000100:00100000:0.0:1789302753.558042:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:27141:x1876219350341120:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302753.558070:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.558100:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc73 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0564 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.558192:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.558210:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.558224:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302753.558270:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x3e:0x0]: returning size 16384 00000004:00200000:0.0:1789302753.558274:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302753.558295:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.558326:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc73 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.558348:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 91 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.558357:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302753.558361:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302753.558365:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516e00, rc 0) 00000100:00100000:0.0:1789302753.558471:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:27141:x1876219350341120:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 431us (694us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.558480:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1001 00000100:00100000:1.0:1789302753.574790:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302753.579464:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350341248 00000100:00100000:2.0:1789302753.579487:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350341248 00000100:00100000:2.0:1789302753.579623:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:2.0:1789302753.579637:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350341248:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302753.579661:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302753.579686:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184dc81 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a056b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302753.579772:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c45f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371118720:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302753.579897:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2a80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302753.586258:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c45f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371118720:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302753.586331:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184dc81 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a056b expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.586363:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184dc81 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a056b expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.586396:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:3.0:1789302753.590906:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350341248:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 11259us (16151us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302753.590919:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:3.0:1789302753.595579:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.596836:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350341504 00000100:00100000:0.0:1789302753.596848:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350341504 00000100:00100000:0.0:1789302753.596907:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1002 00000100:00100000:0.0:1789302753.596917:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:27141:x1876219350341504:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:0.0:1789302753.597182:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].26, 136 off11728 00000100:00100000:0.0:1789302753.597276:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:27141:x1876219350341504:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 360us (1705us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.597285:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1002 00000040:00080000:3.0:1789302753.934778:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302753.934789:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302753.934829:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 27 count 4 00000040:00080000:3.0:1789302753.934836:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302753.935136:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcded80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371118848:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302753.935243:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00010000:00010000:2.0:1789302753.937821:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302753.943421:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcded80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371118848:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302753.953060:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302753.953146:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350341632 00000100:00100000:1.0:1789302753.953162:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350341632 00000100:00100000:1.0:1789302753.953232:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1003 00000100:00100000:1.0:1789302753.953242:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27151:x1876219350341632:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302753.953277:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302753.953417:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27151:x1876219350341632:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 176us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302753.953426:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1003 00000100:00100000:1.0:1789302753.959546:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302753.959644:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350341888 00000100:00100000:0.0:1789302753.959654:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350341888 00000100:00100000:0.0:1789302753.959717:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1004 00000100:00100000:0.0:1789302753.959728:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27151:x1876219350341888:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302753.959749:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302753.959772:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc88 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0572 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.959876:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc8f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.959891:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.959902:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302753.959914:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:0.0:1789302753.960070:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dc88 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0572 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.960088:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0572 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.960098:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302753.960102:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302753.960105:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302753.960182:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27151:x1876219350341888:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 455us (645us total) trans 4294967943 rc 0/0 00000100:00100000:0.0:1789302753.960189:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1004 00000100:00100000:3.0:1789302753.966276:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302753.966869:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350342016 00000100:00100000:1.0:1789302753.966887:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350342016 00000100:00100000:1.0:1789302753.966971:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 44 00000100:00100000:1.0:1789302753.966982:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2348:x1876219350342016:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302753.966997:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302753.967022:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dc8f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0572 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.967043:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:1.0:1789302753.967048:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302753.967105:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2348:x1876219350342016:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 125us (857us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302753.967130:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 44 00000100:00100000:3.0:1789302753.970073:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302753.970389:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350342144 00000100:00100000:1.0:1789302753.970404:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350342144 00000100:00100000:1.0:1789302753.971312:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 264 00000100:00100000:1.0:1789302753.971329:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350342144:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302753.971406:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.971432:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.971442:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302753.971447:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302753.971452:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 4/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.971520:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.971552:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 10 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302753.971572:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371118976:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302753.971683:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1300 going to sleep for 11 seconds 00000100:00100000:0.0:1789302753.975859:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371118976:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302753.975889:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302753.976153:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302753.976217:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350342272 00000100:00100000:1.0:1789302753.976241:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350342272 00002000:00100000:1.0:1789302753.976291:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x15:0x0] (0->157) req@ffff960f3fcdf800 x1876219350342272/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:329/0 lens 648/0 e 0 to 0 dl 1789302764 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302753.976300:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdf800 with x1876219350342272 ext(0->157) 00010000:00010000:1.0:1789302753.976307:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 11 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.976320:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 11 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302753.976327:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3fcdf800 00000100:00100000:1.0:1789302753.976369:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 8 00000100:00100000:1.0:1789302753.976387:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350342272:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302753.976898:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350342272:12345-192.168.203.11@tcp:4:cp.0 Request processed in 512us (753us total) trans 4294967348 rc 0/0 00000100:00100000:1.0:1789302753.976908:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 8 00002000:00100000:1.0:1789302753.976915:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x15:0x0] (0->157) req@ffff960f3fcdf800 x1876219350342272/t4294967348(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:329/0 lens 648/448 e 0 to 0 dl 1789302764 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302753.976923:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdf800 with x1876219350342272 ext(0->157) 00010000:00010000:1.0:1789302753.976928:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 13 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302753.976942:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 13 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302753.976960:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3fcdf800 00000100:00100000:1.0:1789302753.987978:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302753.988286:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350342400 00000100:00100000:3.0:1789302753.988303:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350342400 00010000:00010000:3.0:1789302753.988383:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 13 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302753.988450:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 302 00000100:00100000:3.0:1789302753.988463:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2345:x1876219350342400:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302753.988480:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302753.988556:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 13 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.988573:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 13 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.988605:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000019b17bd5/0x64fe43095184dc49 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a054f expref: 13 pid: 6695 timeout: 690 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.988636:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302753.988642:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302753.988666:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302753.988677:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302753.988786:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302753.988854:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350342400:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 390us (880us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302753.988879:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 302 00010000:00010000:0.0:1789302753.988964:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.988978:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.989016:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.989025:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.989041:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.989049:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302753.989055:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.989064:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.989083:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302753.989091:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000994d4816/0x64fe43095184dc9d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302753.989228:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350342144:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 17903us (19171us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302753.989237:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 264 00000100:00100000:3.0:1789302753.996973:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302753.997032:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350342656 00000100:00100000:1.0:1789302753.997046:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350342656 00000100:00100000:1.0:1789302753.997123:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1005 00000100:00100000:1.0:1789302753.997135:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:27151:x1876219350342656:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302753.997287:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:27151:x1876219350342656:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 153us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302753.997295:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1005 00000100:00100000:3.0:1789302754.004187:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.005279:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350342784 00000100:00100000:2.0:1789302754.005295:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350342784 00000100:00100000:2.0:1789302754.005393:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 265 00000100:00100000:2.0:1789302754.005404:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350342784:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302754.005424:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302754.005449:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0579 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302754.005476:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 158, mtime: 0x6aa697e1, atime: 0x0, ctime: 0x6aa697e1, blocks: 0x1 00010000:00010000:2.0:1789302754.005498:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=591s v=(0 0) 00010000:00010000:2.0:1789302754.005520:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302754.005529:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 12 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.005548:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 12 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.005560:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3a00, rc 0) 00000100:00100000:2.0:1789302754.005630:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350342784:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 227us (1465us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302754.005638:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 265 00000100:00100000:2.0:1789302754.012391:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302754.012528:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350343040 00000100:00100000:3.0:1789302754.012539:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350343040 00000100:00100000:3.0:1789302754.012596:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1006 00000100:00100000:3.0:1789302754.012607:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:27151:x1876219350343040:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302754.012710:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:27151:x1876219350343040:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 105us (335us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302754.012718:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1006 00000100:00100000:3.0:1789302754.017633:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302754.017732:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350343168 00000100:00100000:0.0:1789302754.017748:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350343168 00002000:00100000:0.0:1789302754.017803:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x15:0x0] (0->4095) req@ffff960f3fcdc700 x1876219350343168/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:330/0 lens 488/0 e 0 to 0 dl 1789302765 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302754.017815:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdc700 with x1876219350343168 ext(0->4095) 00002000:00010000:0.0:1789302754.017833:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fcdc700 00000100:00100000:0.0:1789302754.017870:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 95 00000100:00100000:0.0:1789302754.017880:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350343168:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302754.018074:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350343168:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 195us (544us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302754.018083:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 95 00002000:00100000:0.0:1789302754.018090:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x15:0x0] (0->4095) req@ffff960f3fcdc700 x1876219350343168/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:330/0 lens 488/4536 e 0 to 0 dl 1789302765 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302754.018097:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdc700 with x1876219350343168 ext(0->4095) 00002000:00010000:0.0:1789302754.018101:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fcdc700 00000100:00100000:3.0:1789302754.026362:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.026481:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350343296 00000100:00100000:2.0:1789302754.026495:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350343296 00000100:00100000:2.0:1789302754.026608:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1007 00000100:00100000:2.0:1789302754.026619:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:27151:x1876219350343296:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302754.026646:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302754.026674:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcab lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0580 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.026719:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.026737:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.026753:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.026877:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcab lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 47 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.026897:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.026905:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302754.026908:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302754.026913:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3200, rc 0) 00000100:00100000:2.0:1789302754.026991:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27151:x1876219350343296:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 373us (644us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302754.026999:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1007 00000100:00100000:3.0:1789302754.033965:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302754.034048:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350343424 00000100:00100000:1.0:1789302754.034058:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350343424 00000100:00100000:1.0:1789302754.034117:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1008 00000100:00100000:1.0:1789302754.034129:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27151:x1876219350343424:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302754.034161:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302754.034271:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27151:x1876219350343424:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 143us (315us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302754.034279:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1008 00000100:00100000:3.0:1789302754.040760:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302754.040851:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350343552 00000100:00100000:1.0:1789302754.040866:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350343552 00000100:00100000:1.0:1789302754.040930:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 266 00000100:00100000:1.0:1789302754.040940:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350343552:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302754.040990:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041021:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302754.041025:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302754.041031:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041045:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041075:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041085:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041093:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041100:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302754.041105:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041114:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041129:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.041144:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003adbb844/0x64fe43095184dcb9 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302754.041234:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350343552:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 295us (490us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302754.041242:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 266 00000100:00100000:2.0:1789302754.050906:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.050955:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350343808 00000100:00100000:2.0:1789302754.050998:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350343808 00000100:00100000:2.0:1789302754.052579:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1009 00000100:00100000:2.0:1789302754.052608:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27151:x1876219350343808:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302754.053128:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.053154:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.053166:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.053221:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.053239:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 46 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302754.053427:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c45c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371119104:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302754.053900:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302754.064665:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.064678:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c45c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371119104:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302754.064735:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302754.064775:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350343936 00000100:00100000:0.0:1789302754.064787:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350343936 00010000:00010000:0.0:1789302754.064827:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302754.064860:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 303 00000100:00100000:0.0:1789302754.064870:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350343936:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302754.064886:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302754.064894:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.064908:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.064932:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dcb2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0580 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.064956:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302754.064962:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.064971:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.064980:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.065008:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302754.065058:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350343936:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 189us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302754.065066:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 303 00010000:00010000:2.0:1789302754.065276:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065287:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065663:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065673:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065680:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065687:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302754.065702:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065711:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065727:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302754.065731:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302754.065736:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.065743:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dcc0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302754.065942:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27151:x1876219350343808:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 13346us (15045us total) trans 4294967944 rc 0/0 00000100:00100000:2.0:1789302754.065954:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1009 00000040:00080000:0.0:1789302754.066055:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302754.066062:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302754.066715:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1789302754.066726:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302754.081239:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302754.082224:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350344192 00000100:00100000:0.0:1789302754.082241:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350344192 00000100:00100000:0.0:1789302754.082314:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 200 00000100:00100000:0.0:1789302754.082324:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27151:x1876219350344192:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302754.084496:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:27151:x1876219350344192:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 2162us (3264us total) trans 4294967945 rc 0/0 00000100:00100000:0.0:1789302754.084546:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 200 00000040:00080000:1.0:1789302754.709092:0:19629:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302754.709101:0:19629:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302754.709797:0:19629:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 27 count 4 00000040:00080000:1.0:1789302754.709804:0:19629:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302754.793810:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.794082:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350344320 00000100:00100000:2.0:1789302754.794093:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350344320 00000100:00100000:2.0:1789302754.794150:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 267 00000100:00100000:2.0:1789302754.794169:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350344320:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302754.794189:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302754.794207:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0587 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.794233:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302754.794237:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302754.794247:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.794264:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.794276:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3e00, rc 0) 00000100:00100000:2.0:1789302754.794334:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350344320:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 168us (536us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302754.794341:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 267 00000100:00100000:2.0:1789302754.810005:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302754.812789:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350344576 00000100:00100000:3.0:1789302754.812803:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350344576 00000100:00100000:3.0:1789302754.812898:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1010 00000100:00100000:3.0:1789302754.812908:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:27153:x1876219350344576:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302754.812928:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302754.812950:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dcce lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a058e expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.813072:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcd5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.813087:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcd5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.813098:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcd5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302754.813110:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:3.0:1789302754.813227:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dcce lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a058e expref: 94 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.813246:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcd5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a058e expref: 93 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.813253:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:3.0:1789302754.813256:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1789302754.813260:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302754.813326:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:27153:x1876219350344576:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 419us (3330us total) trans 4294967946 rc 0/0 00000100:00100000:3.0:1789302754.813333:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1010 00000100:00100000:3.0:1789302754.827096:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302754.827238:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350344704 00000100:00100000:1.0:1789302754.827252:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350344704 00000100:00100000:1.0:1789302754.827324:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 268 00000100:00100000:1.0:1789302754.827337:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2346:x1876219350344704:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:1.0:1789302754.827408:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.827688:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.827701:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.827726:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302754.827732:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302754.827737:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.827956:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.827978:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 12 pid: 6695 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302754.827998:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371119232:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302754.828122:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.828133:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 11 pid: 18578 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302754.828150:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371119360:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302754.828181:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302754.834866:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.834905:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.834960:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350344832 00000100:00100000:2.0:1789302754.834970:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350344832 00010000:00010000:2.0:1789302754.835018:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 13 pid: 6695 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302754.835049:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f024000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371119232:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302754.835063:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350344960 00000100:00100000:3.0:1789302754.835080:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371119360:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302754.835084:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350344960 00010000:00010000:2.0:1789302754.835106:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 12 pid: 18578 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.835119:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302754.835141:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 304 00000100:00100000:2.0:1789302754.835157:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350344832:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302754.835173:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302754.835195:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 13 pid: 6695 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.835211:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 13 pid: 6695 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302754.835218:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:1.0:1789302754.835242:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350344960:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302754.835252:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003f9f0da5/0x64fe43095184dcc7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0587 expref: 13 pid: 6695 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.835260:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302754.835273:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302754.835329:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b3850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350344832:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 176us (484us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302754.835336:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 304 00010000:00010000:1.0:1789302754.835377:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 12 pid: 18578 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.835399:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 12 pid: 18578 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.835423:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184dca4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0579 expref: 12 pid: 18578 timeout: 691 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.836239:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302754.836246:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302754.836253:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.836335:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302754.836383:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302754.836423:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836449:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836524:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836534:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836540:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836547:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302754.836551:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836561:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836579:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.836587:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184dce3 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302754.836918:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350344960:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1679us (2006us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302754.836930:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:2.0:1789302754.841544:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350344704:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 14095us (14346us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302754.841561:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 268 00000100:00100000:2.0:1789302754.848029:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302754.848103:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350345216 00000100:00100000:0.0:1789302754.848120:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350345216 00000100:00100000:0.0:1789302754.848209:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 201 00000100:00100000:0.0:1789302754.848225:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ef450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:27153:x1876219350345216:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:0.0:1789302754.848320:0:5847:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x1b:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcd5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a058e expref: 93 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302754.850119:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967947 00000001:00080000:0.0:1789302754.855269:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967948 00010000:00010000:0.0:1789302754.855402:0:5847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.855431:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.855448:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 96 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.855458:0:5847:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 5/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.855541:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 96 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.855561:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 96 pid: 6919 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302754.855579:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371119488:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302754.855719:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.855744:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 46 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302754.855755:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371119616:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302754.855785:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302754.867697:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302754.869664:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371119488:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302754.869686:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371119616:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302754.869719:0:5847:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302754.870144:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302754.870219:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350345472 00000100:00100000:2.0:1789302754.870221:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350345344 00000100:00100000:0.0:1789302754.870231:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350345472 00000100:00100000:2.0:1789302754.870232:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350345344 00010000:00010000:2.0:1789302754.870287:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 97 pid: 6919 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.870291:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302754.870350:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:0.0:1789302754.870381:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 307 00000100:00100000:2.0:1789302754.870382:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:2347:x1876219350345344:12345-192.168.203.11@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1789302754.870403:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350345472:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302754.870406:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302754.870419:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 97 pid: 6919 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.870423:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302754.870436:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.870450:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 97 pid: 6919 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.870489:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.874541:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184dc65 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0556 expref: 97 pid: 6919 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.874564:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302754.874570:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.874579:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.874595:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302754.874666:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350345344:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4304us (4528us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302754.874675:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 306 00010000:00010000:0.0:1789302754.877438:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dc34 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0541 expref: 47 pid: 5845 timeout: 691 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.877473:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302754.877524:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.877537:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.877546:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.877599:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302754.877717:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350345472:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 7318us (10033us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302754.877733:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 307 00010000:00010000:3.0:1789302754.878379:0:5847:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.878392:0:5847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302754.878666:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967949 00000001:00080000:3.0:1789302754.885053:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967950 00000001:00080000:3.0:1789302754.885157:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967951 00000100:00100000:3.0:1789302754.885276:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371119744:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1789302754.885386:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302754.885441:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f11e89780 going to sleep for 11 seconds 00000100:00100000:0.0:1789302754.886136:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371119744 00000100:00100000:0.0:1789302754.886148:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371119744 00000100:00100000:0.0:1789302754.886237:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 37 00000100:00100000:0.0:1789302754.886249:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371119744:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1789302754.886380:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371119744:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 121us (982us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302754.886390:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 37 00000100:00080000:3.0:1789302754.887748:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302754.887824:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371119744:0@lo:1000:mdt_rdpg00_001.0 00000001:00080000:3.0:1789302754.888306:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967952 00000100:00100000:3.0:1789302754.888419:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371119872:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:3.0:1789302754.888488:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302754.888561:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302754.888583:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371119872 00000100:00100000:0.0:1789302754.888598:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371119872 00000100:00100000:0.0:1789302754.888655:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 38 00000100:00100000:0.0:1789302754.888666:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371119872:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1789302754.888951:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5847:x1876219371119872:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 285us (462us total) trans 4294967349 rc 0/0 00000100:00100000:0.0:1789302754.888959:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 38 00000100:00100000:3.0:1789302754.889412:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371119872:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:3.0:1789302754.889445:0:5847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.889458:0:5847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.889468:0:5847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.889493:0:5847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302754.889498:0:5847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.889553:0:5847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.889573:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:3.0:1789302754.889577:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:3.0:1789302754.889583:0:5847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.889590:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dcea lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302754.889929:0:5847:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x3f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1789302754.892848:0:5847:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].12, 64 off8984 00000004:00080000:2.0:1789302754.892872:0:5847:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 591903369012ns, next at 592903363727ns, rc = 0 00000001:00080000:2.0:1789302754.892890:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967953 00010000:00010000:2.0:1789302754.892989:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302754.892994:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1789302754.893009:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcd5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a058e expref: 100 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302754.893189:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ef450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:27153:x1876219350345216:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 44968us (45170us total) trans 4294967954 rc 0/0 00000100:00100000:2.0:1789302754.893200:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 201 00000020:00080000:2.0:1789302754.893208:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1789302754.914654:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302754.915123:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350345600 00000100:00100000:3.0:1789302754.915140:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350345600 00000100:00100000:3.0:1789302754.915217:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1011 00000100:00100000:3.0:1789302754.915229:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:27154:x1876219350345600:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302754.915260:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302754.915288:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dcf1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0595 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.915418:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.915445:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.915472:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.915490:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 3/1,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.915496:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.915534:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915763:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184dd06 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915781:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184dd06 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915794:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184dd06 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302754.915839:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00000004:00200000:2.0:1789302754.915844:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302754.915885:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184dd06 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915897:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184dd06 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915917:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915924:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915930:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915936:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302754.915939:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915946:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915957:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:2.0:1789302754.915960:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302754.915965:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915970:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184dcf8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915984:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.915989:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916001:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916007:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302754.916009:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916015:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916023:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x385708b9 (ffff960f0cdf2a00) 00010000:00010000:2.0:1789302754.916025:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302754.916029:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 2/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916034:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dcff lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x385708b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916077:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dcf1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0595 expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916091:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184dd06 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0595 expref: 100 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302754.916098:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03927a00) 00010000:00010000:2.0:1789302754.916100:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302754.916103:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302754.916185:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:27154:x1876219350345600:12345-192.168.203.11@tcp:101:diff.0 Request processed in 958us (1539us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302754.916202:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1011 00000100:00100000:0.0:1789302754.924553:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302754.925654:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350345856 00000100:00100000:3.0:1789302754.925674:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350345856 00000100:00100000:3.0:1789302754.925760:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1012 00000100:00100000:3.0:1789302754.925787:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:27154:x1876219350345856:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302754.925815:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302754.925848:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd0d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a059c expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.925988:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dd14 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.926006:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dd14 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.926021:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dd14 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302754.926035:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:3.0:1789302754.926235:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd0d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a059c expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.926257:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dd14 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a059c expref: 102 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.926267:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302754.926271:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302754.926275:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1600, rc 0) 00000100:00100000:3.0:1789302754.926383:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:27154:x1876219350345856:12345-192.168.203.11@tcp:101:diff.0 Request processed in 611us (1838us total) trans 4294967955 rc 0/0 00000100:00100000:3.0:1789302754.926393:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1012 00000100:00100000:3.0:1789302754.932413:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302754.933813:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350346112 00000100:00100000:1.0:1789302754.933827:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350346112 00000100:00100000:0.0:1789302754.934030:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 45 00000100:00100000:0.0:1789302754.934051:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:27154:x1876219350346112:12345-192.168.203.11@tcp:103:diff.0 00010000:00010000:0.0:1789302754.934068:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1789302754.934092:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dd14 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a059c expref: 102 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.934114:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184dd06 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0595 expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.934126:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302754.934148:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302754.934218:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:27154:x1876219350346112:12345-192.168.203.11@tcp:103:diff.0 Request processed in 169us (1822us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302754.934226:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 45 00000100:00100000:0.0:1789302754.936490:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302754.936644:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350346240 00000100:00100000:3.0:1789302754.936663:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350346240 00000100:00100000:3.0:1789302754.936756:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1013 00000100:00100000:3.0:1789302754.936771:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:27154:x1876219350346240:12345-192.168.203.11@tcp:58:diff.0 00010000:00010000:3.0:1789302754.936932:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.936954:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302754.936969:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302754.937207:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].27, 136 off11864 00000100:00100000:3.0:1789302754.937333:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:27154:x1876219350346240:12345-192.168.203.11@tcp:58:diff.0 Request processed in 564us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302754.937362:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1013 00000100:00100000:1.0:1789302754.946448:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302754.946549:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350346368 00000100:00100000:0.0:1789302754.946564:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350346368 00000100:00100000:0.0:1789302754.946632:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1014 00000100:00100000:0.0:1789302754.946642:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:27154:x1876219350346368:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:0.0:1789302754.946662:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302754.946682:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05a3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.946736:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.946746:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05a3 expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.946776:0:5844:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.946790:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05a3 expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.946803:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:0.0:1789302754.946807:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302754.946811:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05a3 expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302754.946819:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516800, rc 0) 00000100:00100000:0.0:1789302754.946893:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:27154:x1876219350346368:12345-192.168.203.11@tcp:101:diff.0 Request processed in 252us (470us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302754.946900:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1014 00010000:00010000:1.0:1789302754.957003:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302755.084869:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302755.084881:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302755.084928:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 28 count 5 00000040:00080000:0.0:1789302755.084939:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302755.085102:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371120000:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302755.085248:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cfcc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302755.100804:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40e00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371120000:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302755.109433:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.111746:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350346496 00000100:00100000:0.0:1789302755.111766:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350346496 00000100:00100000:0.0:1789302755.111878:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1015 00000100:00100000:0.0:1789302755.111893:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27155:x1876219350346496:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000100:00080000:3.0:1789302755.111917:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000004:00000080:0.0:1789302755.111961:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302755.112135:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27155:x1876219350346496:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 246us (2709us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.112143:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1015 00000100:00100000:1.0:1789302755.117831:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302755.118001:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350346624 00000100:00100000:3.0:1789302755.118011:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350346624 00000100:00100000:3.0:1789302755.118071:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1016 00000100:00100000:3.0:1789302755.118080:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27155:x1876219350346624:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1789302755.118152:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118168:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118181:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118195:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118202:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118208:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118226:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118233:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118266:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302755.118304:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00000004:00200000:3.0:1789302755.118307:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302755.118318:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118328:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118335:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118342:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118348:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302755.118352:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118359:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118379:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302755.118382:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302755.118386:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118392:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184dd30 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118415:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118421:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118427:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118432:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302755.118435:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118448:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118455:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x2b74b2f9 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302755.118457:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302755.118460:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118465:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd37 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x2b74b2f9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118479:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118486:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118492:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118556:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302755.118560:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118566:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118575:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302755.118578:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302755.118581:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.118588:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd3e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302755.118664:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27155:x1876219350346624:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 584us (840us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302755.118673:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1016 00000100:00100000:3.0:1789302755.122609:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302755.130453:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350346880 00000100:00100000:1.0:1789302755.130487:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350346880 00000100:00100000:1.0:1789302755.130575:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1017 00000100:00100000:1.0:1789302755.130587:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:27155:x1876219350346880:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302755.130606:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302755.130627:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dd45 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05aa expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.130726:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.130741:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.130754:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302755.130765:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1b:0x0]: returning size 158 00010000:00010000:1.0:1789302755.130877:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.130885:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.130891:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.130897:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302755.130902:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.132858:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.132896:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:1.0:1789302755.132899:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302755.132916:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.132924:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184dd4c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.133442:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dd45 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05aa expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.133456:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302755.133459:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302755.133463:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6000/0x64fe43095184dd45 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05aa expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.133474:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6000, rc 0) 00000100:00100000:0.0:1789302755.133574:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:27155:x1876219350346880:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2987us (10973us total) trans 4294967956 rc 301/301 00000100:00100000:0.0:1789302755.133583:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1017 00000100:00100000:0.0:1789302755.137547:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302755.137632:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350347008 00000100:00100000:1.0:1789302755.137642:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350347008 00000100:00100000:1.0:1789302755.137704:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1018 00000100:00100000:1.0:1789302755.137714:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:27155:x1876219350347008:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302755.137747:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1b:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302755.137860:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:27155:x1876219350347008:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 148us (321us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.137868:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1018 00000100:00100000:1.0:1789302755.144483:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302755.144616:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350347264 00000100:00100000:3.0:1789302755.144626:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350347264 00000100:00100000:3.0:1789302755.144684:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1019 00000100:00100000:3.0:1789302755.144695:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:27155:x1876219350347264:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302755.144716:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302755.144737:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd5a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05b1 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.145286:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd61 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.145314:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.145325:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd61 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302755.145336:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1c:0x0]: returning size 0 00010000:00010000:3.0:1789302755.145396:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184dd5a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b1 expref: 50 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.145416:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd61 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b1 expref: 49 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.145425:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302755.145427:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302755.145431:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0a00, rc 0) 00000100:00100000:3.0:1789302755.145531:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:27155:x1876219350347264:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 809us (1029us total) trans 4294967957 rc 0/0 00000100:00100000:3.0:1789302755.145538:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1019 00000100:00100000:3.0:1789302755.170638:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.170753:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350347520 00000100:00100000:0.0:1789302755.170763:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350347520 00000100:00100000:0.0:1789302755.170833:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1020 00000100:00100000:0.0:1789302755.170852:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:27155:x1876219350347520:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302755.170918:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302755.170942:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dd61 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b1 expref: 49 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.170963:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302755.170989:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302755.171011:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.171024:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 100 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.171043:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 4/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.171075:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 100 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.171089:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 100 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.171104:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371120128:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302755.172289:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf300 going to sleep for 11 seconds 00000100:00100000:1.0:1789302755.194694:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.194799:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350347648 00000100:00100000:2.0:1789302755.194810:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350347648 00010000:00010000:2.0:1789302755.194867:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 101 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.194914:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 308 00000100:00100000:2.0:1789302755.194925:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:2347:x1876219350347648:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302755.194939:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302755.194946:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 101 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.194960:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 101 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.194982:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:2.0:1789302755.194987:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,1 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.194997:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.195005:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.195025:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302755.195079:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:2347:x1876219350347648:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 155us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302755.195106:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 308 00010000:00010000:0.0:1789302755.195587:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516e00/0x64fe43095184dc7a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3e:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0564 expref: 100 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.195606:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371120128:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302755.195628:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195637:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195644:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195676:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195696:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195704:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195782:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195790:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195797:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195802:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302755.195806:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195814:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195823:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302755.195826:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302755.195830:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 2/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195836:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184dd6f lrc: 0/0,0 mode: --/CW res: [0x200000401:0x3e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195853:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195860:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195866:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195871:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302755.195874:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195880:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195888:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0xea94ac7a (ffff960f03501800) 00010000:00010000:0.0:1789302755.195890:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302755.195893:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.195898:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184dd76 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x3e:0x0].0xea94ac7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.195973:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:27155:x1876219350347520:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 25130us (25343us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302755.195981:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1020 00000100:00080000:3.0:1789302755.196388:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302755.199756:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.200815:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350347776 00000100:00100000:0.0:1789302755.200827:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350347776 00000100:00100000:0.0:1789302755.200887:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1021 00000100:00100000:0.0:1789302755.200914:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:27155:x1876219350347776:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302755.200934:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302755.200959:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dd7d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05b8 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.200996:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.201009:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.201021:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.201113:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dd7d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 50 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.201136:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.201144:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:0.0:1789302755.201147:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302755.201151:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:0.0:1789302755.201220:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:27155:x1876219350347776:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 308us (1473us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.201227:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1021 00000100:00100000:2.0:1789302755.207740:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.209322:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350348032 00000100:00100000:0.0:1789302755.209334:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350348032 00000100:00100000:0.0:1789302755.209400:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1022 00000100:00100000:0.0:1789302755.209427:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:27155:x1876219350348032:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302755.209542:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.209561:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.209573:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302755.210173:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ed8f1960 reply state 00000000b5384bc1 transno 4294967958 ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.210183:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302755.210285:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:27155:x1876219350348032:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 859us (2552us total) trans 4294967958 rc 0/0 00000100:00100000:0.0:1789302755.210293:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1022 00010000:00010000:1.0:1789302755.212410:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.212426:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.212434:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.212439:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302755.212460:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.212469:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.212485:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:1.0:1789302755.212488:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302755.212493:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.212535:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd8b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.223374:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.223537:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350348160 00000100:00100000:0.0:1789302755.223547:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350348160 00000100:00100000:0.0:1789302755.223613:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1023 00000100:00100000:0.0:1789302755.223624:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:27155:x1876219350348160:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302755.223648:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302755.223670:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05bf expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.223741:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 51 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.223755:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 51 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.224039:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:0.0:1789302755.224042:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302755.224045:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120ddc00, rc 0) 00000100:00100000:0.0:1789302755.224109:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:27155:x1876219350348160:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 486us (751us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.224115:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1023 00000100:00100000:2.0:1789302755.231818:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.231986:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350348288 00000100:00100000:0.0:1789302755.231998:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350348288 00000100:00100000:0.0:1789302755.232058:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 202 00000100:00100000:0.0:1789302755.232068:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:27155:x1876219350348288:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:0.0:1789302755.232200:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:27155:x1876219350348288:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 133us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.232208:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 202 00000100:00100000:1.0:1789302755.235243:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.235352:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350348544 00000100:00100000:2.0:1789302755.235361:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350348544 00000100:00100000:2.0:1789302755.235429:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 203 00000100:00100000:2.0:1789302755.235439:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:27155:x1876219350348544:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302755.236265:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:27155:x1876219350348544:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 824us (1037us total) trans 4294967959 rc 0/0 00000100:00100000:2.0:1789302755.236274:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 203 00000100:00100000:2.0:1789302755.268882:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302755.269132:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350348800 00000100:00100000:1.0:1789302755.269142:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350348800 00000100:00100000:1.0:1789302755.269204:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1024 00000100:00100000:1.0:1789302755.269213:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:27155:x1876219350348800:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302755.269320:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:27155:x1876219350348800:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 107us (445us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.269328:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1024 00000100:00100000:3.0:1789302755.278283:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.278416:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350349056 00000100:00100000:0.0:1789302755.278428:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350349056 00000100:00100000:0.0:1789302755.278495:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1025 00000100:00100000:0.0:1789302755.278558:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:27155:x1876219350349056:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302755.278578:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302755.278599:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd99 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05c6 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.278705:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.278721:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 53 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.278745:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.278790:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 53 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.278803:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 53 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.278829:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371120256:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302755.278911:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf300 going to sleep for 11 seconds 00000100:00100000:3.0:1789302755.289297:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.289415:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371120256:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302755.289448:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.289644:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350349184 00000100:00100000:1.0:1789302755.289653:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350349184 00010000:00010000:1.0:1789302755.289701:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 54 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.289740:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 309 00000100:00100000:1.0:1789302755.289750:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2348:x1876219350349184:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302755.289795:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302755.289803:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 54 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.289815:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 54 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.289836:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184dd92 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05bf expref: 54 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.289851:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:1.0:1789302755.289856:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.289866:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.289873:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.289902:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302755.289960:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2348:x1876219350349184:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 211us (672us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.289968:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 309 00010000:00010000:0.0:1789302755.290097:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290106:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302755.290124:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1c:0x0]: returning size 0 00000004:00080000:0.0:1789302755.290302:0:18427:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x15:0x0], index 1: 0 00010000:00010000:0.0:1789302755.290480:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290488:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290546:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290552:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302755.290556:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290563:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290573:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:0.0:1789302755.290575:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302755.290578:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290584:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dda0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290637:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd99 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05c6 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290647:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3e:0x0].0x0 (ffff960f03927f00) 00010000:00010000:0.0:1789302755.290650:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302755.290654:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184dd99 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x3e:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05c6 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.290663:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dda00, rc 0) 00000100:00100000:0.0:1789302755.290692:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:27155:x1876219350349056:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 12190us (12418us total) trans 4294967960 rc 301/301 00000100:00100000:0.0:1789302755.290708:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1025 00000100:00100000:1.0:1789302755.295177:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302755.295266:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350349440 00000100:00100000:3.0:1789302755.295276:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350349440 00000100:00100000:3.0:1789302755.295333:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 204 00000100:00100000:3.0:1789302755.295343:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:27155:x1876219350349440:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302755.296053:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:27155:x1876219350349440:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 709us (882us total) trans 4294967961 rc 0/0 00000100:00100000:3.0:1789302755.296062:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 204 00000100:00100000:2.0:1789302755.304122:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.312430:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350349568 00000100:00100000:0.0:1789302755.312455:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350349568 00000100:00100000:0.0:1789302755.312894:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1026 00000100:00100000:0.0:1789302755.312907:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:27155:x1876219350349568:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302755.312933:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302755.312954:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184ddae lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05cd expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.313033:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184ddb5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.313047:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184ddb5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.313057:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184ddb5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302755.313091:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1c:0x0]: returning size 0 00000004:00200000:0.0:1789302755.313095:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302755.313110:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184ddb5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.313134:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184ddae lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05cd expref: 55 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.313152:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184ddb5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05cd expref: 54 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.313180:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:0.0:1789302755.313182:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1789302755.313186:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516200, rc 0) 00000100:00100000:0.0:1789302755.313258:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:27155:x1876219350349568:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 354us (9145us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.313264:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1026 00000100:00100000:1.0:1789302755.329603:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.329700:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350349696 00000100:00100000:2.0:1789302755.329711:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350349696 00000100:00100000:2.0:1789302755.329781:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 269 00000100:00100000:2.0:1789302755.329791:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350349696:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302755.329809:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302755.329830:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05d4 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302755.329861:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x15:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302755.329884:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=592s v=(0 0) 00010000:00010000:2.0:1789302755.329888:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302755.329897:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.329935:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.329949:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3800, rc 0) 00000100:00100000:2.0:1789302755.330017:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350349696:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 228us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302755.330024:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 269 00000100:00080000:3.0:1789302755.331860:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302755.339233:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.339315:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350349952 00000100:00100000:0.0:1789302755.339325:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350349952 00000100:00100000:0.0:1789302755.339388:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1027 00000100:00100000:0.0:1789302755.339398:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:27155:x1876219350349952:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302755.339544:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:27155:x1876219350349952:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 147us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.339552:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1027 00000100:00100000:2.0:1789302755.341029:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371120384:0@lo:400:kworker.0 00000100:00100000:2.0:1789302755.341095:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302755.341191:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371120384 00000100:00100000:1.0:1789302755.341199:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371120384 00000100:00100000:1.0:1789302755.341239:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 556 00000100:00100000:1.0:1789302755.341262:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371120384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302755.342676:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371120512:0@lo:400:kworker.0 00000100:00100000:3.0:1789302755.342726:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302755.344492:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371120512 00000100:00100000:3.0:1789302755.344516:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371120512 00000100:00100000:3.0:1789302755.344584:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1789302755.344594:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371120512:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302755.344682:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371120512:12345-0@lo:400:kworker.0 Request processed in 90us (1955us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302755.344689:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 384 00000100:00100000:3.0:1789302755.344734:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371120512:0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.346387:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371120384:12345-0@lo:400:kworker.0 Request processed in 5118us (5282us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.346400:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 556 00000100:00100000:0.0:1789302755.347031:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302755.347047:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371120896:0@lo:400:kworker.0 00000100:00100000:0.0:1789302755.347100:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302755.347155:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371121024:0@lo:400:kworker.0 00000100:00100000:0.0:1789302755.347181:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302755.347215:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371121024 00000100:00100000:2.0:1789302755.347223:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371121024 00000100:00100000:1.0:1789302755.347356:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371120640:0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.347396:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302755.347421:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302755.347437:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371120768:0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.347459:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302755.347498:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371120768 00000100:00100000:1.0:1789302755.347523:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371120768 00000100:00100000:1.0:1789302755.347561:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1789302755.347570:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371120768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.347621:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371120768:12345-0@lo:400:kworker.0 Request processed in 51us (159us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.347627:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 376 00000100:00100000:1.0:1789302755.347665:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371120768:0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.347687:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371120896 00000100:00100000:1.0:1789302755.347690:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371120896 00000100:00100000:1.0:1789302755.347716:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371120640 00000100:00100000:1.0:1789302755.347719:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371120640 00000100:00100000:1.0:1789302755.347740:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789302755.347746:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371120896:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.347784:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371120896:12345-0@lo:400:kworker.0 Request processed in 38us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.347790:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00100000:1.0:1789302755.347818:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:1.0:1789302755.347824:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371120640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.347859:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371120640:12345-0@lo:400:kworker.0 Request processed in 35us (463us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.347865:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:3.0:1789302755.347880:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371120640:0@lo:400:kworker.0 00000100:00100000:1.0:1789302755.347887:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371120896:0@lo:400:kworker.0 00000100:00100000:2.0:1789302755.347999:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1789302755.348007:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371121024:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302755.348046:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371121024:12345-0@lo:400:kworker.0 Request processed in 41us (863us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302755.348052:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 377 00000100:00100000:2.0:1789302755.348081:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371121024:0@lo:400:kworker.0 00000100:00100000:2.0:1789302755.348102:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371120384:0@lo:400:kworker.0 00000100:00080000:0.0:1789302755.348343:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302755.348348:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302755.352983:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.353113:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350350080 00000100:00100000:2.0:1789302755.353124:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350350080 00000100:00100000:2.0:1789302755.353198:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 270 00000100:00100000:2.0:1789302755.353208:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:27155:x1876219350350080:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302755.353243:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302755.353263:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05db expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.353300:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.353310:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302755.353315:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302755.353320:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.353662:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.353678:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 11 pid: 13439 timeout: 692 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.353692:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371121152:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302755.353819:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2f00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302755.356848:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371121152:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302755.356883:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.357088:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302755.357214:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:27155:x1876219350350080:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4006us (4239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302755.357224:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 270 00000100:00100000:1.0:1789302755.369480:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.369802:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350350208 00000100:00100000:0.0:1789302755.369812:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350350208 00010000:00010000:0.0:1789302755.369852:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 11 pid: 13439 timeout: 692 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.369927:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 310 00000100:00100000:0.0:1789302755.369937:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350350208:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302755.369953:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302755.369978:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 11 pid: 13439 timeout: 692 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.369992:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 11 pid: 13439 timeout: 692 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.370015:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003d1c4527/0x64fe43095184ddbc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05d4 expref: 11 pid: 13439 timeout: 692 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.370032:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302755.370037:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302755.370045:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.370057:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.370095:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 10 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.370108:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5830:1876219371121280:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302755.370195:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf000 going to sleep for 11 seconds 00000100:00100000:0.0:1789302755.392350:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0001_UUID:5830:1876219371121280:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302755.392388:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302755.392495:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b0c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350350208:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 22557us (23022us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.392550:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 310 00000100:00100000:2.0:1789302755.410392:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302755.410642:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350350336 00000100:00100000:3.0:1789302755.410653:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350350336 00002000:00100000:3.0:1789302755.410876:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x15:0x0] (0->157) req@ffff960e03980000 x1876219350350336/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:331/0 lens 648/0 e 0 to 0 dl 1789302766 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302755.410888:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03980000 with x1876219350350336 ext(0->157) 00002000:00010000:3.0:1789302755.410894:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03980000 00000100:00100000:2.0:1789302755.410989:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 96 00000100:00100000:2.0:1789302755.411001:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350350336:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302755.413997:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350350336:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 2990us (3604us total) trans 4294967354 rc 0/0 00000100:00100000:2.0:1789302755.414013:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 96 00002000:00100000:2.0:1789302755.414024:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x15:0x0] (0->157) req@ffff960e03980000 x1876219350350336/t4294967354(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:331/0 lens 648/448 e 0 to 0 dl 1789302766 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302755.414034:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e03980000 with x1876219350350336 ext(0->157) 00002000:00010000:2.0:1789302755.414040:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e03980000 00000100:00100000:1.0:1789302755.421496:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.425847:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350350464 00000100:00100000:0.0:1789302755.425862:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350350464 00000100:00100000:0.0:1789302755.425938:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1028 00000100:00100000:0.0:1789302755.425949:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:27155:x1876219350350464:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000100:00100000:0.0:1789302755.460407:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:27155:x1876219350350464:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 34449us (38909us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.460421:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1028 00000001:00080000:2.0:1789302755.460713:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967960 is committed 00000001:00080000:2.0:1789302755.460730:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967949 is committed 00000001:00080000:2.0:1789302755.460761:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967961 is committed 00000001:00080000:2.0:1789302755.460770:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967955 is committed 00000004:00080000:2.0:1789302755.460775:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 13 committed 00000020:00080000:2.0:1789302755.460799:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f0ab25f00 refcount 2 committed 0 result 0 batchid 17 00000020:00080000:2.0:1789302755.460803:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10e85d00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f0ab25cc0 00000020:00080000:2.0:1789302755.460806:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10e85a00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3b085000 00000020:00080000:3.0:1789302755.460890:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 17 committed batchid 16 00000020:00080000:3.0:1789302755.461093:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 16 00000100:00100000:2.0:1789302755.461724:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302755.461741:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371121408:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789302755.461795:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302755.461963:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371121408 00000100:00100000:0.0:1789302755.461971:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371121408 00000100:00100000:0.0:1789302755.462021:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:0.0:1789302755.462030:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371121408:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789302755.462053:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:21 count 1 00002000:00010000:0.0:1789302755.462083:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x15:0x0] initial LVB size: 158, mtime: 0x6aa697e1, atime: 0x0, ctime: 0x6aa697e1, blocks: 0x1 00010000:00010000:0.0:1789302755.462091:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 3/0,1 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.462106:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=592s v=(0 0) 00010000:00010000:0.0:1789302755.462110:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302755.462117:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.462152:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.462162:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.462171:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.462189:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302755.463144:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371121408:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1110us (1343us total) trans 4294967350 rc 0/0 00000100:00100000:0.0:1789302755.463154:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00010000:00010000:0.0:1789302755.463179:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302755.463186:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.463195:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302755.463199:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.463209:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.463226:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 1/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.463242:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184ddca lrc: 0/0,0 mode: --/PW res: [0x240000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.463271:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302755.463273:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302755.463275:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789302755.465713:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f41fb8380/2, rc 0, transno 4294967350 00000100:00100000:2.0:1789302755.465749:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371121408:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789302755.468482:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.469000:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350350592 00000100:00100000:2.0:1789302755.469010:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350350592 00000100:00100000:2.0:1789302755.469172:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 272 00000100:00100000:2.0:1789302755.469184:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350350592:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000001:00080000:0.0:1789302755.531866:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967354 is committed 00000100:00100000:2.0:1789302755.532701:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350350592:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 63512us (64219us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302755.532715:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 272 00000100:00100000:3.0:1789302755.542242:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302755.542603:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350350848 00000100:00100000:1.0:1789302755.542623:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350350848 00000100:00100000:1.0:1789302755.542746:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1029 00000100:00100000:3.0:1789302755.543957:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:27155:x1876219350350848:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302755.544042:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302755.544069:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184ddb5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05cd expref: 43 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.544091:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:3.0:1789302755.544095:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302755.544123:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.544137:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.544147:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302755.545279:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b8c87a5f reply state 0000000050fac245 transno 4294967962 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302755.545293:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302755.545426:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27155:x1876219350350848:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 2667us (3191us total) trans 4294967962 rc 0/0 00000100:00100000:3.0:1789302755.545443:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1029 00000100:00100000:3.0:1789302755.552816:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302755.553094:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350350976 00000100:00100000:1.0:1789302755.553113:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350350976 00000100:00100000:1.0:1789302755.553200:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1030 00000100:00100000:1.0:1789302755.553213:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27155:x1876219350350976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302755.553248:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302755.553291:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05e2 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.553382:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 45 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.553409:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 45 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.553916:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:1.0:1789302755.553920:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302755.553925:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7c00, rc 0) 00000100:00100000:1.0:1789302755.554011:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:27155:x1876219350350976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 799us (1202us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.554019:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1030 00010000:00010000:2.0:1789302755.554162:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.554174:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.554184:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.554192:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302755.554198:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.554209:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.554224:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:2.0:1789302755.554227:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302755.554231:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.554238:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ddd1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.558022:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302755.558150:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350351104 00000100:00100000:3.0:1789302755.558165:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350351104 00000100:00100000:3.0:1789302755.558487:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 273 00000100:00100000:3.0:1789302755.558500:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350351104:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1789302755.558964:0:6696:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1789302755.558985:0:6696:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb mtime from disk: 0 -> 1789302753 00002000:00010000:3.0:1789302755.558988:0:6696:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb atime from disk: 0 -> 1789302753 00002000:00010000:3.0:1789302755.558990:0:6696:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb ctime from disk: 0 -> 1789302753 00002000:00010000:3.0:1789302755.558993:0:6696:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x15:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1789302755.559080:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350351104:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 581us (1072us total) trans 4294967355 rc 0/0 00000100:00100000:3.0:1789302755.559089:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 273 00000100:00100000:0.0:1789302755.569151:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.569222:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350351360 00000100:00100000:2.0:1789302755.569236:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350351360 00000100:00100000:2.0:1789302755.569315:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1031 00000100:00100000:2.0:1789302755.569328:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:27155:x1876219350351360:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1789302755.569432:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.569466:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 44 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.569477:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.569545:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 44 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.569562:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 44 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.569580:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371121536:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302755.569701:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f142aaf00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302755.576620:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.576752:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371121536:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302755.576792:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.576958:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350351488 00000100:00100000:1.0:1789302755.576968:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350351488 00010000:00010000:1.0:1789302755.577016:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 45 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.577058:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00100000:1.0:1789302755.577068:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2347:x1876219350351488:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302755.577087:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302755.577094:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 45 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.577108:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 45 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.577131:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184dd84 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05b8 expref: 45 pid: 5844 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.577150:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:1.0:1789302755.577173:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.577185:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.577194:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.577226:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302755.577300:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350351488:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 232us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.577308:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 311 00010000:00010000:2.0:1789302755.578082:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578093:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578145:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578153:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578159:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578165:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302755.578170:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578178:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578191:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:2.0:1789302755.578194:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302755.578199:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.578205:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184dddf lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.578447:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:27155:x1876219350351360:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 9111us (9299us total) trans 0 rc -22/-22 00000100:00100000:2.0:1789302755.578459:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1031 00000100:00100000:2.0:1789302755.585343:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302755.585430:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350351616 00000100:00100000:1.0:1789302755.585453:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350351616 00000100:00100000:1.0:1789302755.585560:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 274 00000100:00100000:1.0:1789302755.585572:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350351616:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302755.585634:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.585668:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.585679:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302755.585685:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302755.585691:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.585738:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 11 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.585770:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 11 pid: 13439 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.585788:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f046680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371121664:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302755.585885:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1300 going to sleep for 11 seconds 00000100:00100000:3.0:1789302755.594609:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.594801:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350351744 00000100:00100000:2.0:1789302755.594815:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350351744 00010000:00010000:2.0:1789302755.594885:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 12 pid: 13439 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.594932:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 312 00000100:00100000:2.0:1789302755.594944:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350351744:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302755.594965:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302755.594995:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 12 pid: 13439 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.595011:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 12 pid: 13439 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.595043:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302755.595048:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302755.595056:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.595067:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.595094:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302755.595162:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350351744:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 219us (568us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302755.595170:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 312 00010000:00010000:1.0:1789302755.597383:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ddc3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x15:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05db expref: 11 pid: 13439 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.597411:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f046680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371121664:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302755.597451:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597464:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597472:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597557:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597567:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597598:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597606:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302755.597611:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597622:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597644:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302755.597651:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005acb256c/0x64fe43095184dde6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302755.597754:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350351616:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 12184us (12425us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.597764:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 274 00000100:00100000:0.0:1789302755.597995:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.598151:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302755.598576:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350352000 00000100:00100000:1.0:1789302755.598586:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350352000 00000100:00100000:1.0:1789302755.598658:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 378 00000100:00100000:1.0:1789302755.598668:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2345:x1876219350352000:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302755.598731:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2345:x1876219350352000:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 65us (737us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302755.598751:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 378 00000100:00100000:2.0:1789302755.598831:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350351872 00000100:00100000:2.0:1789302755.598836:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350351872 00000100:00100000:2.0:1789302755.598879:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 557 00000100:00100000:2.0:1789302755.598896:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350351872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302755.598928:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350351872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (777us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302755.598935:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 557 00000100:00100000:3.0:1789302755.600827:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:0.0:1789302755.601015:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302755.601890:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350352256 00000100:00100000:2.0:1789302755.601907:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350352256 00000100:00100000:2.0:1789302755.601984:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1032 00000100:00100000:2.0:1789302755.601995:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:27155:x1876219350352256:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302755.602102:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.602124:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.602137:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.605782:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.605821:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 44 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.605832:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.605888:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 44 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.605933:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 44 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.605954:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371121792:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302755.606099:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f142aab40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302755.615003:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371121792:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302755.615043:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302755.619035:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.619597:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350352384 00000100:00100000:0.0:1789302755.619611:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350352384 00010000:00010000:0.0:1789302755.619667:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 45 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302755.619713:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 313 00000100:00100000:0.0:1789302755.619724:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2347:x1876219350352384:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302755.619744:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302755.619751:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 45 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.619796:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 45 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.619825:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ddd8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e2 expref: 45 pid: 6919 timeout: 692 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.619849:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:0.0:1789302755.619855:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.619864:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.619873:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.619910:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302755.619978:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350352384:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 255us (957us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.619985:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 313 00010000:00010000:2.0:1789302755.620726:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.620739:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302755.620990:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967963 00000001:00080000:2.0:1789302755.621178:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967964 00000001:00080000:2.0:1789302755.621233:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967965 00000100:00100000:2.0:1789302755.621430:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371121920:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1789302755.621632:0:5845:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302755.621796:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cc54180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302755.629061:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371121920 00000100:00100000:3.0:1789302755.629084:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371121920 00000100:00100000:0.0:1789302755.631604:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 39 00000100:00100000:0.0:1789302755.631632:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371121920:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789302755.631788:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371121920:12345-0@lo:1000:mdt00_002.0 Request processed in 168us (10157us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.631796:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 39 00000100:00100000:2.0:1789302755.631891:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371121920:0@lo:1000:mdt00_002.0 00000001:00080000:2.0:1789302755.632374:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967966 00000100:00100000:2.0:1789302755.632479:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371122048:0@lo:1000:mdt00_002.0 00000100:00100000:2.0:1789302755.632550:0:5845:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302755.643353:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f12195000 going to sleep for 11 seconds 00000100:00100000:1.0:1789302755.643730:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371122048 00000100:00100000:1.0:1789302755.643744:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371122048 00000100:00100000:1.0:1789302755.643816:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 40 00000100:00100000:1.0:1789302755.643829:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371122048:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789302755.644201:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5845:x1876219371122048:12345-0@lo:1000:mdt00_002.0 Request processed in 374us (11650us total) trans 4294967356 rc 0/0 00000100:00100000:1.0:1789302755.644210:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 40 00000100:00100000:3.0:1789302755.644380:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371122048:0@lo:1000:mdt00_002.0 00000001:00080000:3.0:1789302755.644564:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967967 00010000:00010000:3.0:1789302755.644611:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644623:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644632:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644639:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302755.644644:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644653:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644688:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501f00) 00010000:00010000:3.0:1789302755.644692:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302755.644699:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644706:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ddf4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1c:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644745:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644752:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644758:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644763:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302755.644766:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644773:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644780:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:3.0:1789302755.644783:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302755.644792:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05a3 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644805:0:5845:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05a3 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644822:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05a3 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302755.644865:0:5845:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516800/0x64fe43095184dd29 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05a3 expref: 88 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302755.644882:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371122176:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302755.644971:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302755.649742:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371122176:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302755.649789:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649803:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dd22 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649844:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649860:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649875:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649888:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649895:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649901:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649908:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302755.649913:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649920:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649934:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:2.0:1789302755.649956:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302755.649961:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649968:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ddfb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649980:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649986:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649991:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.649997:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302755.650000:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.650007:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.650015:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1b:0x0].0x0 (ffff960f0cdf2f00) 00010000:00010000:2.0:1789302755.650018:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302755.650021:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.650026:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184dded lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302755.650297:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:27155:x1876219350352256:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 48301us (49480us total) trans 4294967968 rc 0/0 00000100:00100000:2.0:1789302755.650317:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1032 00000020:00080000:2.0:1789302755.650325:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:1.0:1789302755.651636:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302755.651655:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302755.652157:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:1.0:1789302755.652167:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302755.652977:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302755.655155:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.655317:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350352512 00000100:00100000:2.0:1789302755.655331:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350352512 00000100:00100000:2.0:1789302755.655404:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350352768 00000100:00100000:2.0:1789302755.655408:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350352768 00000100:00100000:2.0:1789302755.655430:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 205 00000100:00100000:2.0:1789302755.655442:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:27154:x1876219350352512:12345-192.168.203.11@tcp:49:diff.0 00000100:00100000:2.0:1789302755.655636:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:27154:x1876219350352512:12345-192.168.203.11@tcp:49:diff.0 Request processed in 195us (2666us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.655644:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 206 00000100:00100000:0.0:1789302755.655656:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076efc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:27155:x1876219350352768:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302755.655657:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 205 00000004:00080000:0.0:1789302755.656693:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1c:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1789302755.656819:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967969 00010000:00010000:0.0:1789302755.656886:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.656910:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.656926:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.656936:0:5846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.656943:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.656949:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.656972:0:5846:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789302755.657582:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302755.657592:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.657600:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302755.657604:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.657614:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.657629:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1c:0x0].0x0 (ffff960f03501800) 00010000:00010000:2.0:1789302755.657633:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789302755.657742:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.657751:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de02 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302755.657790:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302755.657792:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302755.657794:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302755.659268:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076efc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:27155:x1876219350352768:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 3601us (4112us total) trans 4294967970 rc 0/0 00000100:00100000:0.0:1789302755.659282:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 206 00000020:00080000:0.0:1789302755.659291:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789302755.665426:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302755.666397:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350352896 00000100:00100000:0.0:1789302755.666414:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350352896 00000100:00100000:0.0:1789302755.666499:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 275 00000100:00100000:0.0:1789302755.666546:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:27154:x1876219350352896:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:0.0:1789302755.666572:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302755.666660:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fc03719c/0x64fe43095184de09 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05e9 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302755.666695:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x15:0x0] initial LVB size: 158, mtime: 0x6aa697e1, atime: 0x6aa697e1, ctime: 0x6aa697e1, blocks: 0xb 00010000:00010000:0.0:1789302755.666723:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=592s v=(0 0) 00010000:00010000:0.0:1789302755.666729:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x15:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302755.666741:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fc03719c/0x64fe43095184de09 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e9 expref: 15 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.666759:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000fc03719c/0x64fe43095184de09 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x15:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05e9 expref: 15 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302755.666774:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516e00, rc 0) 00000100:00100000:0.0:1789302755.666852:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:27154:x1876219350352896:12345-192.168.203.11@tcp:101:diff.0 Request processed in 310us (1436us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.666860:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 275 00000100:00100000:1.0:1789302755.676259:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.676389:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350353024 00000100:00100000:2.0:1789302755.676402:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350353024 00002000:00100000:2.0:1789302755.676469:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x15:0x0] (0->4095) req@ffff960f3f046680 x1876219350353024/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:331/0 lens 488/0 e 0 to 0 dl 1789302766 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302755.676483:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f046680 with x1876219350353024 ext(0->4095) 00002000:00010000:2.0:1789302755.676490:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f046680 00000100:00100000:2.0:1789302755.676559:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 97 00000100:00100000:2.0:1789302755.676571:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2347:x1876219350353024:12345-192.168.203.11@tcp:3:diff.0 00000100:00100000:2.0:1789302755.676867:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2347:x1876219350353024:12345-192.168.203.11@tcp:3:diff.0 Request processed in 298us (622us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1789302755.676876:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 97 00002000:00100000:2.0:1789302755.676885:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x15:0x0] (0->4095) req@ffff960f3f046680 x1876219350353024/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:331/0 lens 488/4536 e 0 to 0 dl 1789302766 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302755.676895:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f046680 with x1876219350353024 ext(0->4095) 00002000:00010000:2.0:1789302755.676899:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f046680 00000100:00100000:0.0:1789302755.689395:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302755.694229:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350353280 00000100:00100000:2.0:1789302755.694251:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350353280 00000100:00100000:2.0:1789302755.694411:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 207 00000100:00100000:2.0:1789302755.694425:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:27154:x1876219350353280:12345-192.168.203.11@tcp:35:diff.0 00000100:00100000:2.0:1789302755.694811:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:27154:x1876219350353280:12345-192.168.203.11@tcp:35:diff.0 Request processed in 388us (5427us total) trans 4294967971 rc 0/0 00000100:00100000:2.0:1789302755.694840:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 207 00000100:00100000:2.0:1789302755.854145:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371122304:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302755.854243:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302755.854447:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371122304 00000100:00100000:0.0:1789302755.854457:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371122304 00000100:00100000:0.0:1789302755.854532:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:0.0:1789302755.854544:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371122304:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302755.854573:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 76480512 pending 0 free 3856662528 avail 3853516800 00000100:00100000:0.0:1789302755.862608:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371122304:12345-0@lo:13:osp-pre-1-0.0 Request processed in 8041us (8340us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302755.862628:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1789302755.863727:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371122304:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302755.990045:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302756.370196:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371122432:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302756.370323:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302756.371429:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371122432 00000100:00100000:0.0:1789302756.371439:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371122432 00000100:00100000:0.0:1789302756.371539:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 167 00000100:00100000:0.0:1789302756.371548:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371122432:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302756.371581:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 78184448 pending 1703936 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302756.371638:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371122432:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (1315us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302756.371645:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 167 00000100:00100000:0.0:1789302756.371689:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371122432:0@lo:13:osp-pre-0-0.0 00000100:00080000:2.0:1789302756.371882:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:3.0:1789302756.786034:0:19682:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302756.786042:0:19682:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302756.786106:0:19682:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 28 count 5 00000040:00080000:3.0:1789302756.786111:0:19682:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302757.004894:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302758.007053:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967350 is committed 00000001:00080000:1.0:1789302758.007071:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967348 is committed 00010000:00010000:2.0:1789302758.030845:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302759.054428:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302759.757929:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302759.757939:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302759.758316:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 28 count 3 00000040:00080000:3.0:1789302759.758324:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302760.080651:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302760.460843:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302760.462749:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371122688:0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.462826:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302760.462888:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302760.462893:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371122816:0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.462916:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302760.462950:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371122688 00000100:00100000:2.0:1789302760.462958:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371122688 00000100:00100000:2.0:1789302760.462998:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371122816 00000100:00100000:2.0:1789302760.463001:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371122816 00000100:00100000:2.0:1789302760.463029:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 387 00000100:00100000:2.0:1789302760.463036:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371122688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.463086:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371122688:12345-0@lo:400:kworker.0 Request processed in 50us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.463093:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 387 00000100:00100000:2.0:1789302760.463109:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 388 00000100:00100000:2.0:1789302760.463125:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371122816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.463154:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371122816:12345-0@lo:400:kworker.0 Request processed in 31us (237us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.463159:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 388 00000100:00080000:2.0:1789302760.463179:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302760.463204:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371122816:0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.463213:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371122688:0@lo:400:kworker.0 00000100:00100000:1.0:1789302760.464938:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371122944:0@lo:400:kworker.0 00000100:00100000:1.0:1789302760.464994:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302760.465047:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371122560:0@lo:400:kworker.0 00000100:00100000:1.0:1789302760.465076:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302760.465128:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371122560 00000100:00100000:1.0:1789302760.465135:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371122560 00000100:00100000:1.0:1789302760.465172:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 558 00000100:00100000:1.0:1789302760.465180:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371122560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302760.465223:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371122560:12345-0@lo:400:kworker.0 Request processed in 44us (146us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302760.465240:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 558 00000100:00100000:2.0:1789302760.465776:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371122944 00000100:00100000:2.0:1789302760.465783:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371122944 00000100:00100000:2.0:1789302760.465838:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 379 00000100:00100000:2.0:1789302760.465846:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371122944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.465888:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371122944:12345-0@lo:400:kworker.0 Request processed in 43us (893us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.465894:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 379 00000100:00080000:2.0:1789302760.465912:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302760.466065:0:3302:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967350 gen 2 00000004:00080000:1.0:1789302760.466071:0:3302:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967349 committed transno 4294967350 00000004:00080000:1.0:1789302760.466089:0:3302:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f41fb8380, transno 4294967350 00000100:00100000:1.0:1789302760.466103:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371122944:0@lo:400:kworker.0 00000100:00100000:1.0:1789302760.466113:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371122560:0@lo:400:kworker.0 00000100:00100000:0.0:1789302760.481810:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371123200:0@lo:400:kworker.0 00000100:00100000:0.0:1789302760.481878:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302760.482028:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371123072:0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.482061:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302760.482101:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123072 00000100:00100000:2.0:1789302760.482109:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123072 00000100:00100000:2.0:1789302760.482162:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 389 00000100:00100000:2.0:1789302760.482172:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371123072:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302760.482222:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371123072:12345-0@lo:400:kworker.0 Request processed in 52us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.482229:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 389 00000100:00080000:2.0:1789302760.482250:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302760.482751:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371123072:0@lo:400:kworker.0 00000100:00100000:1.0:1789302760.483046:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123200 00000100:00100000:1.0:1789302760.483052:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123200 00000100:00100000:1.0:1789302760.483090:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1789302760.483098:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371123200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302760.483147:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371123200:12345-0@lo:400:kworker.0 Request processed in 50us (1269us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302760.483153:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 380 00000100:00100000:1.0:1789302760.483187:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371123200:0@lo:400:kworker.0 00000100:00080000:1.0:1789302760.483202:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000001:00080000:3.0:1789302760.545663:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967970 is committed 00000001:00080000:3.0:1789302760.545692:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967971 is committed 00000020:00080000:3.0:1789302760.545703:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f0cc546c0 refcount 2 committed 0 result 0 batchid 18 00000020:00080000:3.0:1789302760.545709:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960e02228b00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f0cc54180 00000020:00080000:3.0:1789302760.545713:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960e02228800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f09816800 00000020:00080000:3.0:1789302760.545758:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 17 batchid updated 00000020:00080000:1.0:1789302760.546009:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 18 committed batchid 17 00000020:00080000:1.0:1789302760.546217:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 17 00000100:00100000:3.0:1789302760.723496:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302760.723589:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302760.723615:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302760.723649:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302760.723669:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302760.723683:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302760.723719:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302760.723893:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350353920 00000100:00100000:2.0:1789302760.723904:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350353920 00000100:00100000:2.0:1789302760.723966:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350353536 00000100:00100000:2.0:1789302760.723969:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350353536 00000100:00100000:2.0:1789302760.724001:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:2.0:1789302760.724010:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2347:x1876219350353920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302760.724082:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2347:x1876219350353920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 74us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.724092:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:2.0:1789302760.724108:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 391 00000100:00100000:2.0:1789302760.724113:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:2347:x1876219350353536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302760.724137:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:2347:x1876219350353536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (468us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.724142:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 391 00000100:00100000:2.0:1789302760.724158:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350353664 00000100:00100000:2.0:1789302760.724161:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350353664 00000100:00100000:2.0:1789302760.724180:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354176 00000100:00100000:2.0:1789302760.724183:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354176 00000100:00100000:2.0:1789302760.724194:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350353792 00000100:00100000:2.0:1789302760.724196:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350353792 00000100:00100000:2.0:1789302760.724205:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354048 00000100:00100000:2.0:1789302760.724206:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354048 00000100:00100000:2.0:1789302760.724217:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 381 00000100:00100000:2.0:1789302760.724222:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350353664:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302760.724243:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350353664:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (655us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.724250:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 381 00000100:00100000:2.0:1789302760.724276:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 382 00000100:00100000:2.0:1789302760.724282:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350354176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302760.724307:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350354176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (658us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.724313:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 382 00000100:00100000:2.0:1789302760.724323:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00100000:2.0:1789302760.724342:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2347:x1876219350353792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302760.724364:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2347:x1876219350353792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (679us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.724370:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00100000:2.0:1789302760.724381:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 384 00000100:00100000:2.0:1789302760.724387:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350354048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302760.724408:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350354048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (689us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302760.724425:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 384 00000100:00100000:1.0:1789302760.724929:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350353408 00000100:00100000:1.0:1789302760.724935:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350353408 00000100:00100000:1.0:1789302760.724970:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 559 00000100:00100000:1.0:1789302760.724977:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350353408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302760.725015:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350353408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (1529us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302760.725020:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 559 00000100:00080000:3.0:1789302760.725361:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302760.725366:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302760.725368:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302760.972901:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302760.972923:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371123328:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302760.972995:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302760.973583:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123328 00000100:00100000:1.0:1789302760.973602:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123328 00000100:00100000:1.0:1789302760.973656:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789302760.973666:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371123328:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302760.973696:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 76480512 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302760.973748:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371123328:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (752us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302760.973755:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:1.0:1789302760.973798:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371123328:0@lo:13:osp-pre-1-0.0 00000001:00080000:0.0:1789302761.049867:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967356 is committed 00000001:00080000:0.0:1789302761.049898:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967355 is committed 00010000:00010000:2.0:1789302761.100939:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302761.484925:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371123456:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302761.484992:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302761.485162:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123456 00000100:00100000:0.0:1789302761.485170:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123456 00000100:00100000:0.0:1789302761.485238:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 169 00000100:00100000:0.0:1789302761.485246:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371123456:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302761.485278:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 76480512 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302761.485330:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371123456:12345-0@lo:13:osp-pre-0-0.0 Request processed in 73us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302761.485337:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 169 00000100:00100000:1.0:1789302761.486892:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371123456:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302762.129382:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302762.743420:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302762.743616:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354304 00000100:00100000:0.0:1789302762.743626:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354304 00000100:00100000:0.0:1789302762.743698:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1033 00000100:00100000:0.0:1789302762.743710:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27134:x1876219350354304:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1789302762.743827:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:27134:x1876219350354304:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 119us (415us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302762.743835:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1033 00000100:00080000:3.0:1789302762.744044:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302763.149915:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302764.172737:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302764.537281:0:19883:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12h: Restore a released file implicitly from a second node ========================================================== 08:32:42 (1789302762) 00010000:00010000:2.0:1789302765.197018:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302765.580921:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371123584:0@lo:400:kworker.0 00000100:00100000:0.0:1789302765.581039:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302765.581277:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371123840:0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.581327:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302765.581361:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302765.581376:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371123712:0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.581399:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302765.581436:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123840 00000100:00100000:3.0:1789302765.581446:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123840 00000100:00100000:3.0:1789302765.581529:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123712 00000100:00100000:3.0:1789302765.581533:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123712 00000100:00100000:3.0:1789302765.581572:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789302765.581582:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371123840:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.581643:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371123840:12345-0@lo:400:kworker.0 Request processed in 62us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302765.581650:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 392 00000100:00100000:3.0:1789302765.581669:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:3.0:1789302765.581675:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371123712:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.581718:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371123712:12345-0@lo:400:kworker.0 Request processed in 44us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302765.581742:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00080000:3.0:1789302765.581760:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302765.581795:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371123712:0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.581807:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371123840:0@lo:400:kworker.0 00000100:00100000:1.0:1789302765.581917:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123584 00000100:00100000:1.0:1789302765.581922:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123584 00000100:00100000:1.0:1789302765.581962:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 560 00000100:00100000:1.0:1789302765.581970:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371123584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.582085:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371123968:0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.582125:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302765.582166:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371123968 00000100:00100000:3.0:1789302765.582172:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371123968 00000100:00100000:3.0:1789302765.582210:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 385 00000100:00100000:3.0:1789302765.582217:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371123968:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302765.582252:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371123968:12345-0@lo:400:kworker.0 Request processed in 37us (127us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302765.582258:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 385 00000100:00080000:3.0:1789302765.582273:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302765.582629:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371123968:0@lo:400:kworker.0 00000100:00100000:2.0:1789302765.597761:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302765.597860:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371124096:0@lo:400:kworker.0 00000100:00100000:2.0:1789302765.597964:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302765.598144:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371124096 00000100:00100000:2.0:1789302765.598154:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371124096 00000100:00100000:2.0:1789302765.598220:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:2.0:1789302765.598229:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371124096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302765.598288:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371124096:12345-0@lo:400:kworker.0 Request processed in 60us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302765.598841:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00080000:2.0:1789302765.598873:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302765.598918:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371124096:0@lo:400:kworker.0 00000100:00100000:2.0:1789302765.598981:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371124224:0@lo:400:kworker.0 00000100:00100000:2.0:1789302765.599020:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302765.601756:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371124224 00000100:00100000:0.0:1789302765.601843:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371124224 00000100:00100000:0.0:1789302765.601930:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 386 00000100:00100000:0.0:1789302765.601943:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371124224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302765.609007:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371123584:12345-0@lo:400:kworker.0 Request processed in 27027us (27957us total) trans 0 rc 0/0 00000100:00080000:0.0:1789302765.609008:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302765.609026:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 560 00000100:00100000:0.0:1789302765.609701:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371123584:0@lo:400:kworker.0 00000100:00100000:1.0:1789302765.610544:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371124224:12345-0@lo:400:kworker.0 Request processed in 8599us (11516us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302765.610559:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 386 00000100:00100000:2.0:1789302765.612213:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967356 gen 2 00000004:00080000:2.0:1789302765.612225:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967356 committed transno 4294967356 00000100:00100000:2.0:1789302765.612260:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371124224:0@lo:400:kworker.0 00000020:00080000:1.0:1789302765.667341:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 18 batchid updated 00000020:00080000:1.0:1789302765.667378:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 18 00000020:00080000:1.0:1789302765.667392:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 18 00000100:00100000:2.0:1789302765.836144:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302765.836196:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302765.836237:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302765.836259:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302765.836280:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302765.836306:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302765.836358:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302765.836396:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354688 00000100:00100000:2.0:1789302765.836411:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354688 00000100:00100000:0.0:1789302765.836618:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354432 00000100:00100000:0.0:1789302765.836623:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354432 00000100:00100000:0.0:1789302765.836670:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 561 00000100:00100000:0.0:1789302765.836682:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350354432:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302765.836787:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350354432:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 106us (656us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302765.836796:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 561 00000100:00100000:0.0:1789302765.836862:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350355200 00000100:00100000:0.0:1789302765.836867:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350355200 00000100:00100000:2.0:1789302765.836986:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354816 00000100:00100000:2.0:1789302765.836991:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354816 00000100:00100000:3.0:1789302765.837057:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354944 00000100:00100000:3.0:1789302765.837062:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354944 00000100:00100000:3.0:1789302765.837098:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350354560 00000100:00100000:3.0:1789302765.837101:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350354560 00000100:00100000:3.0:1789302765.837124:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 395 00000100:00100000:3.0:1789302765.837132:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2346:x1876219350354944:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302765.837174:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2346:x1876219350354944:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (935us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302765.837181:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 395 00000100:00100000:3.0:1789302765.837196:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 396 00000100:00100000:3.0:1789302765.837218:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:2346:x1876219350354560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302765.837244:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:2346:x1876219350354560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (962us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302765.837251:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 396 00000100:00080000:3.0:1789302765.837448:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302765.837454:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302765.837454:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350355072 00000100:00080000:3.0:1789302765.837457:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302765.837460:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350355072 00000100:00100000:0.0:1789302765.837551:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 387 00000100:00100000:0.0:1789302765.837560:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350354688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302765.837735:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350354688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 50us (1412us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302765.837756:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 387 00000100:00100000:0.0:1789302765.837776:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 388 00000100:00100000:0.0:1789302765.837783:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350355200:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302765.837814:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350355200:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (1552us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302765.837820:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 388 00000100:00080000:0.0:1789302765.837863:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302765.838072:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 389 00000100:00100000:2.0:1789302765.838083:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2346:x1876219350354816:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302765.838128:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2346:x1876219350354816:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 48us (1822us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302765.838135:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 389 00000100:00100000:0.0:1789302765.839981:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:0.0:1789302765.840004:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350355072:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302765.840096:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350355072:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 103us (3737us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302765.840104:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:1.0:1789302766.095844:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f047480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371124352:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302766.097164:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302766.098298:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371124352 00000100:00100000:2.0:1789302766.098308:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371124352 00000100:00100000:2.0:1789302766.098369:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1789302766.098379:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371124352:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302766.098414:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 76480512 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302766.098474:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371124352:12345-0@lo:13:osp-pre-1-0.0 Request processed in 97us (1314us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302766.098481:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:2.0:1789302766.098546:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f047480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371124352:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302766.220929:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302766.394731:0:19934:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: SKIP: sanity-hsm test_12h Need 2 or more clients, have 1 00000100:00100000:0.0:1789302766.607486:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371124480:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302766.607569:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302766.607638:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371124480 00000100:00100000:0.0:1789302766.607646:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371124480 00000100:00100000:0.0:1789302766.607688:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 171 00000100:00100000:0.0:1789302766.607697:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371124480:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302766.607731:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 76480512 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302766.607774:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371124480:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302766.607781:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 171 00000100:00100000:0.0:1789302766.607826:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef31880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371124480:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302767.245119:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302768.269190:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302768.275749:0:19982:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12m: Archive/release/implicit restore ========================================================== 08:32:46 (1789302766) 00000100:00100000:0.0:1789302768.627882:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302768.627990:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350355328 00000100:00100000:2.0:1789302768.627999:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350355328 00000100:00100000:2.0:1789302768.628117:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1034 00000100:00100000:2.0:1789302768.628129:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:28056:x1876219350355328:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302768.628239:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:28056:x1876219350355328:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 112us (364us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302768.628246:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1034 00000100:00080000:0.0:1789302768.632080:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302768.650606:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.650689:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350355456 00000100:00100000:0.0:1789302768.650703:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350355456 00000100:00100000:0.0:1789302768.650794:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1035 00000100:00100000:0.0:1789302768.650804:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:28057:x1876219350355456:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302768.650823:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.650843:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184de10 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05f0 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.650903:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.650918:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302768.650929:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302768.650931:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.650950:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.650959:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.650967:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651185:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651205:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651212:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651218:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.651223:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651231:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651243:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302768.651246:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.651251:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651256:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de17 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651286:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651293:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651306:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651311:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.651321:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651327:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651336:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff960f03501800) 00010000:00010000:0.0:1789302768.651338:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.651341:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651347:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de1e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651381:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184de10 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05f0 expref: 90 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651391:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302768.651392:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.651396:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184de10 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05f0 expref: 90 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.651404:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:0.0:1789302768.651472:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:28057:x1876219350355456:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 668us (884us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302768.651479:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1035 00000100:00100000:1.0:1789302768.657194:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302768.657294:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350355712 00000100:00100000:2.0:1789302768.657310:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350355712 00000100:00100000:2.0:1789302768.657390:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1036 00000100:00100000:2.0:1789302768.657419:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:28057:x1876219350355712:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:2.0:1789302768.657479:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302768.657547:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184db69 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0502 expref: 89 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.657568:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302768.657573:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302768.657779:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.657794:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 40 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.657802:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.657837:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 40 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.657851:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 40 pid: 5844 timeout: 705 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302768.657864:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f757480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371124608:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302768.657962:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302768.659491:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f757480 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371124608:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302768.659559:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302768.660564:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302768.660764:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350355840 00000100:00100000:2.0:1789302768.660776:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350355840 00010000:00010000:2.0:1789302768.660819:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 41 pid: 5844 timeout: 705 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302768.660862:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 314 00000100:00100000:2.0:1789302768.660872:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2347:x1876219350355840:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302768.660888:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302768.660895:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 41 pid: 5844 timeout: 705 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.660908:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 41 pid: 5844 timeout: 705 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.660931:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184dba8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0517 expref: 41 pid: 5844 timeout: 705 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.660959:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302768.660964:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.660973:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.660981:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.661022:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302768.661094:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:2347:x1876219350355840:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 223us (542us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302768.661101:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 314 00010000:00010000:3.0:1789302768.661710:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.661719:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.661744:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.661754:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.661763:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302768.664324:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 16384 00010000:00010000:3.0:1789302768.664341:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.664351:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.664369:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.664374:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.664379:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.664387:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.664403:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03501200) 00010000:00010000:3.0:1789302768.664406:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.664410:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.664416:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184de25 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302768.664855:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000083a7f226 reply state 00000000167dbdcf transno 4294967972 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302768.664865:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302768.665316:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:28057:x1876219350355712:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 7896us (8144us total) trans 4294967972 rc 0/0 00000100:00100000:3.0:1789302768.665325:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1036 00010000:00010000:3.0:1789302768.671896:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.671921:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.671954:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.671961:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.671967:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.671975:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.671995:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff960f0cdf2400) 00010000:00010000:3.0:1789302768.671999:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.672005:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.672012:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de2c lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302768.675624:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.675732:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350355968 00000100:00100000:3.0:1789302768.675747:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350355968 00000100:00100000:3.0:1789302768.675816:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1037 00000100:00100000:3.0:1789302768.675829:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+89:28058:x1876219350355968:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302768.675860:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302768.675893:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184de33 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05f7 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.675981:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184de3a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.676018:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184de3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.676032:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184de3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302768.676078:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302768.676083:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302768.676106:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184de3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.676133:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184de33 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05f7 expref: 91 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.676153:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184de3a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05f7 expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.676163:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302768.676167:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302768.676171:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302768.676307:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:28058:x1876219350355968:12345-192.168.203.11@tcp:101:bash.0 Request processed in 479us (696us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302768.677499:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1037 00000100:00100000:1.0:1789302768.692398:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.692553:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356096 00000100:00100000:0.0:1789302768.692567:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356096 00000100:00100000:0.0:1789302768.692636:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1038 00000100:00100000:0.0:1789302768.692647:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+90:28058:x1876219350356096:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302768.692693:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.692720:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de41 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a05fe expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.692808:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.692826:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.692840:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.692858:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.692866:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.692872:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.692991:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184de56 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693000:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184de56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693008:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184de56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302768.693041:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 16384 00000004:00200000:0.0:1789302768.693051:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302768.693077:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184de56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693093:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693100:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693108:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693115:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.693120:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693129:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693143:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302768.693147:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.693153:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693160:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de48 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693175:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693182:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693199:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693203:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.693206:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693212:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693220:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff960f03501800) 00010000:00010000:0.0:1789302768.693223:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.693226:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693232:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de4f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693284:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de41 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05fe expref: 92 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693301:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517400/0x64fe43095184de56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05fe expref: 91 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.693309:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302768.693311:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302768.693314:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517400, rc 0) 00000100:00100000:0.0:1789302768.693405:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28058:x1876219350356096:12345-192.168.203.11@tcp:101:bash.0 Request processed in 759us (1021us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302768.693422:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1038 00000100:00100000:1.0:1789302768.704568:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.707783:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356224 00000100:00100000:3.0:1789302768.707805:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356224 00000100:00100000:0.0:1789302768.708050:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1039 00000100:00100000:0.0:1789302768.708063:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28058:x1876219350356224:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302768.708089:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.708117:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de5d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0605 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708213:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708231:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708246:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708423:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708434:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708441:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708554:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708563:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708592:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708599:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.708604:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708613:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708627:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302768.708630:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.708636:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708643:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184de64 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708660:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708667:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708673:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708679:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.708682:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708700:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.708709:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f03501800) 00010000:00010000:0.0:1789302768.708711:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.708714:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.709613:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184de6b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.709686:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de5d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0605 expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.709699:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302768.709702:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.709707:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184de5d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0605 expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.709718:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302768.709826:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28058:x1876219350356224:12345-192.168.203.11@tcp:101:bash.0 Request processed in 1764us (5272us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302768.709836:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1039 00000100:00100000:1.0:1789302768.727933:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.728021:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356352 00000100:00100000:3.0:1789302768.728030:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356352 00000100:00100000:3.0:1789302768.728087:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1040 00000100:00100000:3.0:1789302768.728097:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+40:28059:x1876219350356352:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302768.728147:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302768.728171:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184de72 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a060c expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.728235:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.728249:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.728277:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302768.728313:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302768.728317:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302768.728335:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.728370:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184de72 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 42 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.728391:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 41 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.728399:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302768.728402:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302768.728405:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1c00, rc 0) 00000100:00100000:3.0:1789302768.728465:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:28059:x1876219350356352:12345-192.168.203.11@tcp:101:rm.0 Request processed in 369us (548us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302768.728480:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1040 00000100:00100000:2.0:1789302768.733950:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.734025:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356480 00000100:00100000:3.0:1789302768.734036:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356480 00000100:00100000:3.0:1789302768.734102:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1041 00000100:00100000:3.0:1789302768.734113:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:28059:x1876219350356480:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302768.734144:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302768.734166:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184de80 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0613 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734232:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734246:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734268:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734292:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734300:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734307:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734414:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184de95 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734452:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184de95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734459:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184de95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302768.734486:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 16384 00000004:00200000:3.0:1789302768.734490:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302768.734555:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184de95 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734567:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734574:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734581:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734587:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.734591:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734598:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734610:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302768.734613:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.734618:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734636:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184de87 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734651:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734657:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734663:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734667:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.734671:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734677:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734684:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c7f40eb (ffff960f0cdf2400) 00010000:00010000:3.0:1789302768.734687:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.734690:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734696:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184de8e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c7f40eb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734727:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184de80 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0613 expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734741:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0000/0x64fe43095184de95 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0613 expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.734756:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302768.734758:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789302768.734761:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0000, rc 0) 00000100:00100000:3.0:1789302768.734814:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28059:x1876219350356480:12345-192.168.203.11@tcp:101:rm.0 Request processed in 703us (874us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302768.734822:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1041 00000100:00100000:1.0:1789302768.739695:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.739788:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356608 00000100:00100000:3.0:1789302768.739798:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356608 00000100:00100000:3.0:1789302768.739852:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1042 00000100:00100000:3.0:1789302768.739861:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28059:x1876219350356608:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302768.739881:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302768.739902:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184de9c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a061a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740222:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740236:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740247:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740261:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740268:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740600:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740653:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740660:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740668:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740674:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.740679:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740686:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740698:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302768.740701:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.740706:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740712:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184dea3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740725:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740731:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740743:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740747:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.740750:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740755:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740762:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f0cdf2600) 00010000:00010000:3.0:1789302768.740764:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.740767:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740773:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184deaa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740805:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184de9c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a061a expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740815:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302768.740817:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.740820:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184de9c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a061a expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.740828:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302768.741064:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28059:x1876219350356608:12345-192.168.203.11@tcp:101:rm.0 Request processed in 1202us (1376us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302768.741080:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1042 00000100:00100000:1.0:1789302768.745070:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.745155:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356736 00000100:00100000:3.0:1789302768.745165:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356736 00000100:00100000:3.0:1789302768.745223:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1043 00000100:00100000:3.0:1789302768.745233:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28059:x1876219350356736:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302768.745252:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302768.745280:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184deb1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0621 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745342:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745354:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745364:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745378:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745385:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745391:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745451:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745476:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745483:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745488:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.745492:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745531:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745543:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302768.745546:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.745550:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745556:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184deb8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745571:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745576:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745581:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745585:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.745593:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745599:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745606:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f0cdf2700) 00010000:00010000:3.0:1789302768.745608:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.745611:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745616:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184debf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745648:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184deb1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0621 expref: 43 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745657:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302768.745659:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.745663:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184deb1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0621 expref: 43 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.745671:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0a00, rc 0) 00000100:00100000:3.0:1789302768.745731:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28059:x1876219350356736:12345-192.168.203.11@tcp:101:rm.0 Request processed in 500us (669us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302768.745737:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1043 00000100:00100000:2.0:1789302768.756720:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.756817:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356864 00000100:00100000:0.0:1789302768.756828:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356864 00000100:00100000:0.0:1789302768.756947:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1044 00000100:00100000:0.0:1789302768.756958:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28060:x1876219350356864:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302768.756997:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.757018:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dec6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0628 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757085:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757099:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757110:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757125:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757144:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757151:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757199:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757206:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757213:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757219:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.757228:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757236:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757248:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302768.757251:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.757256:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757263:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184decd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757289:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757295:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757300:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757305:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.757307:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757312:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757319:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f03501800) 00010000:00010000:0.0:1789302768.757321:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.757331:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757337:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095184ded4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757381:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dec6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0628 expref: 43 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757392:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302768.757394:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.757398:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184dec6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0628 expref: 43 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.757406:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302768.757475:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28060:x1876219350356864:12345-192.168.203.11@tcp:101:cp.0 Request processed in 519us (764us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302768.757483:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1044 00000100:00100000:1.0:1789302768.760683:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.760760:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350356992 00000100:00100000:3.0:1789302768.760769:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350356992 00000100:00100000:3.0:1789302768.760822:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1045 00000100:00100000:3.0:1789302768.760832:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28060:x1876219350356992:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302768.760868:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302768.760888:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dedb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a062f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.760973:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.760987:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.760998:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761012:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761019:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761026:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761079:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761086:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761093:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761099:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.761108:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761118:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761141:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302768.761144:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.761149:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761156:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184dee2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761174:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761180:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761186:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761191:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302768.761193:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761199:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761206:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f0cdf2700) 00010000:00010000:3.0:1789302768.761208:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.761217:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761222:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dee9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761253:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dedb lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a062f expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761262:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302768.761264:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302768.761268:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184dedb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a062f expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.761443:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0a00, rc 0) 00000100:00100000:3.0:1789302768.761597:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28060:x1876219350356992:12345-192.168.203.11@tcp:101:cp.0 Request processed in 765us (922us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302768.761605:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1045 00000100:00100000:2.0:1789302768.764865:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302768.765152:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350357248 00000100:00100000:3.0:1789302768.765163:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350357248 00000100:00100000:3.0:1789302768.765438:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1046 00000100:00100000:3.0:1789302768.765448:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28060:x1876219350357248:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302768.765465:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302768.765483:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184def0 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0636 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.765628:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.765663:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.765674:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.765689:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 3/0,1 mode: --/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.765696:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302768.765703:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302768.766896:0:5845:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x16:0x0], index 0: 0 00010000:00010000:2.0:1789302768.767253:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767266:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767286:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767319:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767327:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767334:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767340:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302768.767344:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767352:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767440:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:2.0:1789302768.767443:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302768.767449:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767456:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184def7 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302768.767470:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000adb81fd5 reply state 00000000a3429017 transno 4294967973 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302768.767478:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302768.767573:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184def0 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767588:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 44 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302768.767595:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:2.0:1789302768.767597:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302768.767600:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2a00, rc 0) 00000100:00100000:2.0:1789302768.767703:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28060:x1876219350357248:12345-192.168.203.11@tcp:101:cp.0 Request processed in 2257us (2846us total) trans 4294967973 rc 0/0 00000100:00100000:2.0:1789302768.767711:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1046 00010000:00010000:1.0:1789302768.770875:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.770890:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.770898:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.770903:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302768.770907:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.770914:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.770930:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f0cdf2600) 00010000:00010000:1.0:1789302768.770933:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302768.770938:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 2/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.770959:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184defe lrc: 0/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302768.773183:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.773243:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350357376 00000100:00100000:0.0:1789302768.773253:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350357376 00000100:00080000:3.0:1789302768.773430:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302768.773468:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 276 00000100:00100000:0.0:1789302768.773479:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350357376:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302768.773497:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.773547:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a063d expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302768.773574:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302768.773595:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=605s v=(0 0) 00010000:00010000:0.0:1789302768.773615:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302768.773624:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.773642:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.773656:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302768.773708:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350357376:12345-192.168.203.11@tcp:101:cp.0 Request processed in 231us (533us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302768.773715:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 276 00000100:00100000:2.0:1789302768.780669:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.780863:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350357504 00000100:00100000:0.0:1789302768.780873:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350357504 00000100:00100000:0.0:1789302768.780937:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:0.0:1789302768.780946:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:28060:x1876219350357504:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302768.780962:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.780987:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0644 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.781018:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.781028:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302768.781033:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302768.781038:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.781090:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.781104:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 11 pid: 13439 timeout: 705 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302768.781118:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371124736:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302768.781216:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61000 going to sleep for 11 seconds 00000100:00100000:3.0:1789302768.785615:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.785791:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371124736:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302768.785824:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.785843:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc 0) 00000100:00100000:0.0:1789302768.785907:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:28060:x1876219350357504:12345-192.168.203.11@tcp:101:cp.0 Request processed in 4961us (5245us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302768.785916:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:1.0:1789302768.788608:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350357632 00000100:00100000:1.0:1789302768.788622:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350357632 00010000:00010000:1.0:1789302768.788672:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 11 pid: 13439 timeout: 705 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302768.788785:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:1.0:1789302768.788796:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350357632:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302768.788812:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302768.788834:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 11 pid: 13439 timeout: 705 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.788849:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 11 pid: 13439 timeout: 705 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.788872:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003ce814be/0x64fe43095184df13 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a063d expref: 11 pid: 13439 timeout: 705 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.788889:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302768.788893:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302768.788902:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.788945:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.788994:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302768.789008:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f047480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371124864:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302768.789095:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdba80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302768.792658:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f047480 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371124864:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302768.792686:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302768.792744:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350357632:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 3949us (7138us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302768.792754:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:3.0:1789302768.799713:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302768.799827:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350357888 00000100:00100000:1.0:1789302768.799837:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350357888 00000100:00100000:1.0:1789302768.799899:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00100000:1.0:1789302768.799909:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28060:x1876219350357888:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302768.803763:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28060:x1876219350357888:12345-192.168.203.11@tcp:35:cp.0 Request processed in 3845us (4049us total) trans 4294967974 rc 0/0 00000100:00100000:0.0:1789302768.803776:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 208 00000100:00100000:3.0:1789302768.866266:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.866537:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350358016 00000100:00100000:0.0:1789302768.866548:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350358016 00000100:00100000:0.0:1789302768.866610:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1047 00000100:00100000:0.0:1789302768.866619:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28063:x1876219350358016:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302768.866657:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.866679:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184df21 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a064b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866756:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866770:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866781:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866796:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866804:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866810:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866894:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866901:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866908:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302768.866951:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00000004:00200000:0.0:1789302768.866961:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302768.866982:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.866994:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867001:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867007:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867014:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867021:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.867025:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867032:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867043:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302768.867046:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.867050:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867057:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df28 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867083:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867098:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867105:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867109:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302768.867112:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867118:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867125:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f03501f00) 00010000:00010000:0.0:1789302768.867128:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302768.867131:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867137:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df2f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867171:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184df21 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867185:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.867193:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302768.867195:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302768.867204:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:0.0:1789302768.867295:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28063:x1876219350358016:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 676us (1045us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302768.867301:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1047 00000100:00100000:1.0:1789302768.872732:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302768.872818:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350358144 00000100:00100000:2.0:1789302768.872828:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350358144 00000100:00100000:2.0:1789302768.872913:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 278 00000100:00100000:2.0:1789302768.872923:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350358144:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302768.872950:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302768.872968:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c810bd51/0x64fe43095184df3d lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0652 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302768.873022:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371124992:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302768.873117:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098eacc0 going to sleep for 11 seconds 00000100:00080000:3.0:1789302768.873774:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:1.0:1789302768.877019:0:8571:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb size: 0 -> 1516 00002000:00010000:1.0:1789302768.877028:0:8571:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb mtime: 0 -> 1789302768 00002000:00010000:1.0:1789302768.877031:0:8571:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb atime: 0 -> 1789302768 00002000:00010000:1.0:1789302768.877033:0:8571:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x16:0x0] updating lvb ctime: 0 -> 1789302768 00000100:00100000:1.0:1789302768.877205:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371124992:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302768.877243:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c810bd51/0x64fe43095184df3d lrc: 1/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0652 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.877357:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c810bd51/0x64fe43095184df3d lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0652 expref: 12 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.877431:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:1.0:1789302768.877496:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350358144:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4574us (4773us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302768.877556:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 278 00000100:00100000:1.0:1789302768.885762:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.885839:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350358272 00000100:00100000:0.0:1789302768.885849:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350358272 00000100:00100000:0.0:1789302768.885909:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1048 00000100:00100000:0.0:1789302768.885919:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28063:x1876219350358272:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302768.885947:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.885967:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184df44 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0659 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.886034:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.886048:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.886059:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302768.886103:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x40:0x0]: returning size 16384 00000004:00200000:0.0:1789302768.886107:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302768.886123:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.886150:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184df44 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 94 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.886170:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 93 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.886195:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:0.0:1789302768.886197:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302768.886201:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516200, rc 0) 00000100:00100000:0.0:1789302768.886286:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28063:x1876219350358272:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 354us (517us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302768.886293:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1048 00000100:00100000:1.0:1789302768.890962:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302768.891057:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350358400 00000100:00100000:2.0:1789302768.891068:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350358400 00000100:00100000:2.0:1789302768.891127:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 279 00000100:00100000:2.0:1789302768.891137:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350358400:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302768.891155:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302768.891173:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006bdfc70e/0x64fe43095184df52 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0660 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302768.891234:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f757800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371125120:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302768.891472:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098eaf00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302768.895723:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f757800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371125120:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302768.895762:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006bdfc70e/0x64fe43095184df52 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0660 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.895784:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006bdfc70e/0x64fe43095184df52 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0660 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302768.895796:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:1.0:1789302768.895892:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350358400:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4756us (4939us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302768.895901:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 279 00000100:00100000:1.0:1789302768.901591:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302768.901687:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350358656 00000100:00100000:2.0:1789302768.901696:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350358656 00000100:00100000:2.0:1789302768.901790:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1049 00000100:00100000:2.0:1789302768.901801:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28063:x1876219350358656:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:2.0:1789302768.902194:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].28, 136 off12000 00000100:00100000:2.0:1789302768.902596:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28063:x1876219350358656:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 794us (1012us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302768.902604:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1049 00000040:00080000:1.0:1789302768.973012:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302768.973022:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302768.973091:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 29 count 4 00000040:00080000:1.0:1789302768.973098:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302768.973388:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371125248:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302768.973536:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdb3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302768.978934:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371125248:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302768.981756:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.982085:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350358784 00000100:00100000:0.0:1789302768.982096:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350358784 00000100:00100000:0.0:1789302768.982192:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1050 00000100:00100000:0.0:1789302768.982203:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28073:x1876219350358784:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302768.982241:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302768.982366:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28073:x1876219350358784:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 165us (864us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302768.982375:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1050 00000100:00100000:1.0:1789302768.990462:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302768.990745:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350359040 00000100:00100000:0.0:1789302768.990760:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350359040 00000100:00100000:0.0:1789302768.990836:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1051 00000100:00100000:0.0:1789302768.990849:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28073:x1876219350359040:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302768.990876:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302768.990903:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184df59 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0667 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.991038:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df60 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.991075:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df60 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.991091:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df60 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302768.991107:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:0.0:1789302768.991288:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095184df59 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0667 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.991311:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df60 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0667 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302768.991320:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302768.991323:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302768.991675:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:0.0:1789302768.991897:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28073:x1876219350359040:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1049us (1448us total) trans 4294967975 rc 0/0 00000100:00100000:0.0:1789302768.991907:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1051 00000100:00100000:2.0:1789302769.014352:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.015777:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350359168 00000100:00100000:3.0:1789302769.015791:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350359168 00000100:00100000:3.0:1789302769.015874:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 46 00000100:00100000:3.0:1789302769.015884:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2346:x1876219350359168:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302769.015900:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302769.015938:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095184df60 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0667 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.015974:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302769.015980:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302769.016039:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2346:x1876219350359168:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 155us (1707us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302769.016047:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 46 00000100:00100000:0.0:1789302769.023729:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.023866:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350359296 00000100:00100000:3.0:1789302769.023877:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350359296 00000100:00100000:3.0:1789302769.023945:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 280 00000100:00100000:3.0:1789302769.023955:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350359296:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302769.024001:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.024021:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.024030:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x16:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302769.024034:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302769.024039:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 4/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.024072:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 10 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.024085:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 10 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302769.024122:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371125376:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302769.024208:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d19c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302769.030414:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.030543:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371125376:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302769.030549:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350359424 00000100:00100000:1.0:1789302769.030568:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350359424 00010000:00010000:3.0:1789302769.030574:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00100000:1.0:1789302769.030645:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x16:0x0] (0->1515) req@ffff960f3f7c8700 x1876219350359424/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:345/0 lens 2008/0 e 0 to 0 dl 1789302780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302769.030658:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7c8700 with x1876219350359424 ext(0->1515) 00010000:00010000:1.0:1789302769.030669:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 11 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.030688:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 11 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302769.030706:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3f7c8700 00000100:00100000:1.0:1789302769.030757:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 9 00000100:00100000:1.0:1789302769.030768:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350359424:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302769.031436:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350359424:12345-192.168.203.11@tcp:4:cp.0 Request processed in 667us (1030us total) trans 4294967351 rc 0/0 00000100:00100000:1.0:1789302769.031446:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 9 00002000:00100000:1.0:1789302769.031455:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x16:0x0] (0->1515) req@ffff960f3f7c8700 x1876219350359424/t4294967351(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:345/0 lens 2008/448 e 0 to 0 dl 1789302780 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302769.031465:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7c8700 with x1876219350359424 ext(0->1515) 00010000:00010000:1.0:1789302769.031480:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 13 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.031530:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 13 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302769.031539:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3f7c8700 00000100:00100000:0.0:1789302769.045558:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.045831:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350359552 00000100:00100000:3.0:1789302769.045842:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350359552 00010000:00010000:3.0:1789302769.045885:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 13 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302769.045949:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 316 00000100:00100000:3.0:1789302769.045959:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2346:x1876219350359552:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302769.045972:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302769.045995:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 13 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.046009:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 13 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.046033:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df1a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0644 expref: 13 pid: 6696 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.046050:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302769.046070:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302769.046076:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.046086:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.046162:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302769.046213:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b1050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350359552:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 255us (665us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302769.046221:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 316 00010000:00010000:0.0:1789302769.052378:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052400:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052466:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052474:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052482:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052490:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302769.052495:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052526:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052546:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.052554:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000cf67e601/0x64fe43095184df6e lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302769.052652:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350359296:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 28694us (28927us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302769.052672:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 280 00000100:00100000:0.0:1789302769.075386:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302769.075567:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350359808 00000100:00100000:0.0:1789302769.075577:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350359808 00000100:00100000:0.0:1789302769.075627:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1052 00000100:00100000:0.0:1789302769.075638:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28073:x1876219350359808:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302769.075744:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28073:x1876219350359808:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 107us (366us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302769.075752:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1052 00000100:00100000:3.0:1789302769.085193:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302769.087217:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350359936 00000100:00100000:0.0:1789302769.087230:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350359936 00000100:00100000:0.0:1789302769.087639:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00100000:0.0:1789302769.087651:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350359936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302769.087669:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302769.087692:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a066e expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302769.087719:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 1516, mtime: 0x6aa697f0, atime: 0x0, ctime: 0x6aa697f0, blocks: 0x1 00010000:00010000:0.0:1789302769.087740:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=606s v=(0 0) 00010000:00010000:0.0:1789302769.087744:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302769.087753:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.087769:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.087802:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc 0) 00000100:00100000:0.0:1789302769.087937:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350359936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 289us (2753us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302769.087945:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00100000:2.0:1789302769.110459:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302769.110814:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350360192 00000100:00100000:1.0:1789302769.110825:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350360192 00000100:00100000:1.0:1789302769.110885:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1053 00000100:00100000:1.0:1789302769.110898:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28073:x1876219350360192:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302769.110999:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28073:x1876219350360192:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 104us (548us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302769.111007:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1053 00000100:00100000:0.0:1789302769.121745:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.122078:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350360320 00000100:00100000:3.0:1789302769.122088:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350360320 00002000:00100000:3.0:1789302769.122134:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x16:0x0] (0->4095) req@ffff960f3ef7fb80 x1876219350360320/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:345/0 lens 488/0 e 0 to 0 dl 1789302780 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302769.122145:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7fb80 with x1876219350360320 ext(0->4095) 00002000:00010000:3.0:1789302769.122151:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ef7fb80 00000100:00100000:3.0:1789302769.122186:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 98 00000100:00100000:3.0:1789302769.122196:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350360320:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302769.122939:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350360320:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 741us (1201us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302769.122949:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 98 00002000:00100000:3.0:1789302769.122957:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x16:0x0] (0->4095) req@ffff960f3ef7fb80 x1876219350360320/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:345/0 lens 488/4536 e 0 to 0 dl 1789302780 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302769.122990:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7fb80 with x1876219350360320 ext(0->4095) 00002000:00010000:3.0:1789302769.122994:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ef7fb80 00000100:00100000:1.0:1789302769.155829:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.155907:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350360448 00000100:00100000:3.0:1789302769.155917:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350360448 00000100:00100000:3.0:1789302769.155974:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1054 00000100:00100000:3.0:1789302769.155984:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28073:x1876219350360448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302769.156003:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302769.156025:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184df7c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0675 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.156059:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.156072:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.156083:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.156184:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184df7c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.156200:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.156209:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302769.156213:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302769.156232:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302769.156300:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28073:x1876219350360448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 318us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302769.156307:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1054 00000100:00100000:2.0:1789302769.162190:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.163627:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350360576 00000100:00100000:3.0:1789302769.163639:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350360576 00000100:00100000:3.0:1789302769.163915:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1055 00000100:00100000:3.0:1789302769.163929:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28073:x1876219350360576:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302769.164371:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302769.164790:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28073:x1876219350360576:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 861us (2605us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302769.164800:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1055 00000100:00100000:3.0:1789302769.175585:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302769.176177:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350360704 00000100:00100000:0.0:1789302769.176187:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350360704 00000100:00100000:0.0:1789302769.176249:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:0.0:1789302769.176260:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350360704:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302769.176316:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 3/1,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176334:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302769.176338:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302769.176343:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176357:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176386:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176394:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176415:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176423:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302769.176428:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176437:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176453:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.176461:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df8a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302769.176560:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350360704:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 302us (985us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302769.176568:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:1.0:1789302769.179561:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.179639:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350360960 00000100:00100000:3.0:1789302769.179649:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350360960 00000100:00100000:3.0:1789302769.179706:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1056 00000100:00100000:3.0:1789302769.179717:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28073:x1876219350360960:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302769.179792:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.179811:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.179820:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.179856:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.179883:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 47 pid: 6919 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302769.179898:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371125504:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302769.179976:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1e40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302769.184998:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302769.185032:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350361088 00000100:00100000:1.0:1789302769.185039:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350361088 00000100:00100000:3.0:1789302769.185053:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371125504:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302769.185075:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 48 pid: 6919 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185080:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302769.185118:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 317 00000100:00100000:1.0:1789302769.185126:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350361088:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302769.185141:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302769.185147:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 48 pid: 6919 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.185174:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 48 pid: 6919 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.185193:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184df83 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0675 expref: 48 pid: 6919 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.185209:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:1.0:1789302769.185214:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.185224:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.185232:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.185256:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302769.185307:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350361088:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 181us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302769.185315:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 317 00010000:00010000:3.0:1789302769.185323:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185331:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185589:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185598:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185606:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185612:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302769.185617:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185636:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185648:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302769.185652:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302769.185656:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.185663:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184df91 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302769.185820:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28073:x1876219350360960:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 6103us (6267us total) trans 4294967976 rc 0/0 00000100:00100000:3.0:1789302769.185832:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1056 00000040:00080000:1.0:1789302769.186257:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302769.186262:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302769.186388:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 29 count 2 00000040:00080000:1.0:1789302769.186394:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302769.190256:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302769.191184:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350361344 00000100:00100000:2.0:1789302769.191195:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350361344 00000100:00100000:2.0:1789302769.191256:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 209 00000100:00100000:2.0:1789302769.191275:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28073:x1876219350361344:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302769.191522:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28073:x1876219350361344:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 256us (1283us total) trans 4294967977 rc 0/0 00000100:00100000:2.0:1789302769.191530:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 209 00010000:00010000:0.0:1789302769.292864:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302769.858666:0:20031:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302769.858674:0:20031:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302769.858709:0:20031:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 29 count 2 00000040:00080000:0.0:1789302769.858716:0:20031:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302769.947191:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302769.947675:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350361472 00000100:00100000:0.0:1789302769.947695:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350361472 00000100:00100000:0.0:1789302769.947785:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:0.0:1789302769.947796:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350361472:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302769.947815:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302769.947834:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a067c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.947860:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302769.947864:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302769.947872:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.947889:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.947900:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:0.0:1789302769.947970:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350361472:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 175us (788us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302769.947977:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:3.0:1789302769.958218:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302769.958333:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350361728 00000100:00100000:3.0:1789302769.958349:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350361728 00000100:00100000:3.0:1789302769.958401:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1057 00000100:00100000:3.0:1789302769.958412:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28074:x1876219350361728:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302769.958432:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302769.958454:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184df9f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0683 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.958616:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184dfa6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.958641:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184dfa6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.958653:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184dfa6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302769.958667:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:3.0:1789302769.958792:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184df9f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0683 expref: 96 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.958811:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184dfa6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0683 expref: 95 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302769.958820:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302769.958823:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1789302769.958826:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1400, rc 0) 00000100:00100000:3.0:1789302769.958881:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:28074:x1876219350361728:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 471us (674us total) trans 4294967978 rc 0/0 00000100:00100000:3.0:1789302769.958888:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1057 00000100:00100000:3.0:1789302769.963687:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302769.963769:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350361856 00000100:00100000:0.0:1789302769.963779:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350361856 00000100:00100000:0.0:1789302769.963840:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:0.0:1789302769.963850:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350361856:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:0.0:1789302769.963891:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.963911:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.963950:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.963958:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302769.963963:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302769.963968:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.964004:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.964019:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 12 pid: 6696 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302769.964036:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371125632:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302769.964206:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.964217:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 11 pid: 6697 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302769.964227:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371125760:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302769.964253:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2f00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302769.973943:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302769.973979:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350361984 00000100:00100000:0.0:1789302769.973988:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350361984 00010000:00010000:0.0:1789302769.974030:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 13 pid: 6696 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302769.974067:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 318 00000100:00100000:0.0:1789302769.974077:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350361984:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302769.974093:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302769.974115:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 13 pid: 6696 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.974131:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 13 pid: 6696 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.974156:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302769.974204:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350361984:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 129us (270us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302769.974212:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 318 00010000:00010000:3.0:1789302769.977595:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000feaa4f08/0x64fe43095184df98 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a067c expref: 12 pid: 6696 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302769.977617:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371125632:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302769.977648:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371125760:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302769.977674:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302769.979316:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302769.979450:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350362112 00000100:00100000:2.0:1789302769.979460:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350362112 00010000:00010000:2.0:1789302769.979556:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 12 pid: 6697 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302769.979598:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 319 00000100:00100000:2.0:1789302769.979610:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350362112:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302769.979625:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302769.979648:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 12 pid: 6697 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302769.979661:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 12 pid: 6697 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302769.979684:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe43095184df75 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a066e expref: 12 pid: 6697 timeout: 706 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302769.979720:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302769.979725:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302769.979730:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302769.979740:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302769.979772:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302769.979822:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350362112:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 214us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302769.979829:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 319 00010000:00010000:0.0:1789302769.980026:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980037:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980061:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980076:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980085:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980092:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302769.980097:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980106:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980118:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302769.980125:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006438736a/0x64fe43095184dfb4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302769.980185:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350361856:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 16337us (16509us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302769.980194:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:3.0:1789302769.987160:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302769.987249:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350362368 00000100:00100000:1.0:1789302769.987259:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350362368 00000100:00100000:1.0:1789302769.987329:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 210 00000100:00100000:1.0:1789302769.987349:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:28074:x1876219350362368:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:1.0:1789302769.987425:0:12976:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x1d:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184dfa6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0683 expref: 95 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789302769.987655:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967979 00000001:00080000:1.0:1789302769.988137:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967980 00010000:00010000:1.0:1789302769.988203:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.988218:0:12976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 47 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.988227:0:12976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 98 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.988234:0:12976:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 5/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.988283:0:12976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 98 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.988301:0:12976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 98 pid: 5844 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302769.988316:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371125888:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302769.988419:0:12976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 47 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302769.988429:0:12976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 47 pid: 5845 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302769.988447:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267f480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371126016:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302769.988472:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f02cd9900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302770.004615:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.005085:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371125888:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302770.008407:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350362496 00000100:00100000:3.0:1789302770.008421:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350362496 00010000:00010000:3.0:1789302770.008476:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 99 pid: 5844 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302770.008536:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:3.0:1789302770.008546:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:2348:x1876219350362496:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302770.008563:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302770.008570:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 99 pid: 5844 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.008584:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 99 pid: 5844 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.008606:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095184df36 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a064b expref: 99 pid: 5844 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.008622:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302770.008628:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.008636:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.008654:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00080000:2.0:1789302770.009034:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302770.013687:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:2348:x1876219350362496:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 5130us (9068us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.013704:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:1.0:1789302770.015194:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.015233:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350362624 00000100:00100000:1.0:1789302770.015242:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350362624 00010000:00010000:1.0:1789302770.015297:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 48 pid: 5845 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.015330:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:1.0:1789302770.015346:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2346:x1876219350362624:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302770.015359:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302770.015365:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 48 pid: 5845 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.015380:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 48 pid: 5845 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.015401:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:1.0:1789302770.015407:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.015417:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.015425:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.015457:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302770.015532:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2346:x1876219350362624:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 187us (341us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.015539:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00080000:1.0:1789302770.015561:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302770.016047:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184df05 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0636 expref: 47 pid: 5845 timeout: 706 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.016075:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267f480 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371126016:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302770.016097:0:12976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.016107:0:12976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.016113:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302770.016231:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967981 00000001:00080000:0.0:1789302770.016458:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967982 00000001:00080000:0.0:1789302770.016554:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967983 00000100:00100000:0.0:1789302770.016641:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371126144:0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:0.0:1789302770.016692:0:12976:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302770.016722:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f011ca540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302770.017432:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371126144 00000100:00100000:1.0:1789302770.017441:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371126144 00000100:00080000:0.0:1789302770.017499:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302770.017516:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 41 00000100:00100000:1.0:1789302770.017526:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371126144:12345-0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:1.0:1789302770.017614:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371126144:12345-0@lo:1000:mdt_rdpg00_002.0 Request processed in 90us (922us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.017621:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 41 00000100:00100000:0.0:1789302770.017742:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371126144:0@lo:1000:mdt_rdpg00_002.0 00000001:00080000:0.0:1789302770.018165:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967984 00000100:00100000:0.0:1789302770.018229:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371126272:0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:0.0:1789302770.018274:0:12976:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302770.018303:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302770.018331:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371126272 00000100:00100000:1.0:1789302770.018337:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371126272 00000100:00100000:1.0:1789302770.018374:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 42 00000100:00100000:1.0:1789302770.018381:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371126272:12345-0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:1.0:1789302770.018551:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:12976:x1876219371126272:12345-0@lo:1000:mdt_rdpg00_002.0 Request processed in 170us (276us total) trans 4294967352 rc 0/0 00000100:00100000:1.0:1789302770.018558:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 42 00000100:00100000:0.0:1789302770.019133:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371126272:0@lo:1000:mdt_rdpg00_002.0 00010000:00010000:0.0:1789302770.019158:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.019183:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.019190:0:12976:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.019195:0:12976:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302770.019200:0:12976:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.019207:0:12976:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.019221:0:12976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302770.019223:0:12976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1789302770.019228:0:12976:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.019234:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184dfbb lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302770.019444:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x41:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1789302770.019562:0:12976:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].13, 64 off9048 00000004:00080000:0.0:1789302770.019574:0:12976:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 607030071004ns, next at 608030068605ns, rc = 0 00000001:00080000:0.0:1789302770.019581:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967985 00010000:00010000:0.0:1789302770.019616:0:12976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302770.019620:0:12976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302770.019626:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184dfa6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0683 expref: 102 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.019725:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28074:x1876219350362368:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 32386us (32575us total) trans 4294967986 rc 0/0 00000100:00100000:0.0:1789302770.019733:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 210 00000020:00080000:0.0:1789302770.019738:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:1.0:1789302770.040351:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.040604:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350362880 00000100:00100000:3.0:1789302770.040614:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350362880 00000100:00100000:3.0:1789302770.040676:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1058 00000100:00100000:3.0:1789302770.040688:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28075:x1876219350362880:12345-192.168.203.11@tcp:101:cmp.0 00010000:00010000:3.0:1789302770.040713:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302770.040734:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfc2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a068a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.040806:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.040822:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.040833:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.040849:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.040857:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.040864:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041014:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dfd7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041024:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dfd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041031:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dfd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302770.041041:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:3.0:1789302770.041134:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041142:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041149:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041155:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302770.041159:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041165:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041178:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302770.041181:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302770.041185:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041202:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dfc9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041216:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041223:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041229:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041233:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302770.041236:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041241:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041249:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x1dee485d (ffff960f0cdf2700) 00010000:00010000:3.0:1789302770.041252:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302770.041255:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041269:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dfd0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x1dee485d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041332:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfc2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a068a expref: 104 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041634:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dfd7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a068a expref: 103 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.041641:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f0cdf2400) 00010000:00010000:3.0:1789302770.041644:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302770.041646:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302770.041737:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:28075:x1876219350362880:12345-192.168.203.11@tcp:101:cmp.0 Request processed in 1051us (1404us total) trans 4294967987 rc 0/0 00000100:00100000:3.0:1789302770.041744:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1058 00000100:00100000:0.0:1789302770.049443:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.051046:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350363136 00000100:00100000:3.0:1789302770.051060:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350363136 00000100:00100000:2.0:1789302770.051190:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 47 00000100:00100000:2.0:1789302770.051201:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:28075:x1876219350363136:12345-192.168.203.11@tcp:103:cmp.0 00010000:00010000:2.0:1789302770.051216:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302770.051239:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184dfd7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a068a expref: 103 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.051272:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f0cdf2400) 00010000:00010000:2.0:1789302770.051285:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302770.051359:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:28075:x1876219350363136:12345-192.168.203.11@tcp:103:cmp.0 Request processed in 159us (1924us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302770.051367:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 47 00000100:00100000:0.0:1789302770.059840:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302770.059902:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350363264 00000100:00100000:2.0:1789302770.059912:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350363264 00000100:00100000:2.0:1789302770.059975:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1059 00000100:00100000:2.0:1789302770.059985:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:28075:x1876219350363264:12345-192.168.203.11@tcp:58:cmp.0 00010000:00010000:2.0:1789302770.060079:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.060095:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.060107:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789302770.060283:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].29, 136 off12136 00000100:00100000:2.0:1789302770.060379:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:28075:x1876219350363264:12345-192.168.203.11@tcp:58:cmp.0 Request processed in 394us (546us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302770.060387:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1059 00000100:00100000:1.0:1789302770.063624:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.063705:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350363392 00000100:00100000:3.0:1789302770.063715:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350363392 00000100:00100000:3.0:1789302770.063770:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1060 00000100:00100000:3.0:1789302770.063781:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:28075:x1876219350363392:12345-192.168.203.11@tcp:101:cmp.0 00010000:00010000:3.0:1789302770.063799:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302770.063819:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0691 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.063880:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.063891:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 4/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0691 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.063910:0:18427:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.063951:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0691 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.063968:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927700) 00010000:00010000:3.0:1789302770.063972:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302770.063976:0:18427:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 3/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0691 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.063983:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1600, rc 0) 00000100:00100000:3.0:1789302770.064180:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:28075:x1876219350363392:12345-192.168.203.11@tcp:101:cmp.0 Request processed in 400us (562us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.064188:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1060 00000040:00080000:1.0:1789302770.188986:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302770.188997:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302770.189040:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 30 count 3 00000040:00080000:1.0:1789302770.189050:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302770.189177:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d880 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371126400:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302770.189435:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f02cd9cc0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302770.196450:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d880 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371126400:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302770.201747:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.202024:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350363520 00000100:00100000:3.0:1789302770.202034:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350363520 00000100:00100000:3.0:1789302770.202097:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1061 00000100:00100000:3.0:1789302770.202107:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28076:x1876219350363520:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302770.202143:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302770.202242:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28076:x1876219350363520:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 136us (505us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.202250:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1061 00000100:00100000:1.0:1789302770.211295:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.211457:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350363648 00000100:00100000:3.0:1789302770.211467:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350363648 00000100:00100000:3.0:1789302770.211546:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1062 00000100:00100000:3.0:1789302770.211558:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28076:x1876219350363648:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1789302770.211644:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211661:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211672:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211686:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211693:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211699:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211719:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211727:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211733:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302770.211767:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00000004:00200000:3.0:1789302770.211771:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302770.211784:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211793:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211820:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211827:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211833:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302770.211837:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211845:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211856:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302770.211859:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302770.211864:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211870:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184dff3 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211882:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211887:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211893:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211898:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302770.211906:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211912:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211932:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x3e15321 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302770.211935:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302770.211938:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211943:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184dffa lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x3e15321 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211957:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211962:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211967:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211971:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302770.211974:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211979:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.211986:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927700) 00010000:00010000:3.0:1789302770.211988:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302770.211997:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.212003:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e001 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302770.212070:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28076:x1876219350363648:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 514us (784us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.212078:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1062 00000100:00100000:0.0:1789302770.217281:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.217371:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350363904 00000100:00100000:3.0:1789302770.217380:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350363904 00000100:00100000:3.0:1789302770.217442:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1063 00000100:00100000:3.0:1789302770.217453:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28076:x1876219350363904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302770.217473:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302770.217492:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e008 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0698 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217632:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217648:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217659:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302770.217671:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1d:0x0]: returning size 1516 00010000:00010000:3.0:1789302770.217769:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217777:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217799:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217819:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302770.217823:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217831:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217842:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927700) 00010000:00010000:3.0:1789302770.217845:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302770.217849:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217856:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e00f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217900:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e008 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0698 expref: 49 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217910:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302770.217913:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302770.217929:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e008 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0698 expref: 49 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.217940:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302770.218135:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28076:x1876219350363904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 685us (863us total) trans 4294967988 rc 301/301 00000100:00100000:3.0:1789302770.218152:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1063 00000100:00100000:0.0:1789302770.227464:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302770.227717:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350364032 00000100:00100000:2.0:1789302770.227727:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350364032 00000100:00100000:2.0:1789302770.227787:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1064 00000100:00100000:2.0:1789302770.227796:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28076:x1876219350364032:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302770.227837:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x1d:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302770.227946:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28076:x1876219350364032:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 148us (489us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302770.227954:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1064 00000100:00100000:2.0:1789302770.234835:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.235669:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350364288 00000100:00100000:1.0:1789302770.235681:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350364288 00000100:00100000:1.0:1789302770.235791:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1065 00000100:00100000:1.0:1789302770.235803:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28076:x1876219350364288:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302770.235824:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302770.235855:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e01d lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a069f expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.236659:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e024 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.236676:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e024 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.236688:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e024 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302770.236698:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 0 00010000:00010000:1.0:1789302770.236746:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e01d lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a069f expref: 51 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.236767:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e024 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a069f expref: 50 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.236775:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302770.236778:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302770.236782:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6800, rc 0) 00000100:00100000:1.0:1789302770.236906:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28076:x1876219350364288:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1105us (2079us total) trans 4294967989 rc 0/0 00000100:00100000:1.0:1789302770.236921:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1065 00000100:00100000:0.0:1789302770.249166:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.252448:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350364544 00000100:00100000:3.0:1789302770.252465:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350364544 00000100:00100000:0.0:1789302770.255991:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1066 00000100:00100000:0.0:1789302770.256008:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28076:x1876219350364544:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302770.256075:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302770.256104:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e024 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a069f expref: 50 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.256125:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:0.0:1789302770.256136:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302770.256163:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.256189:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 102 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.256198:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.256425:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 102 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.256441:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 102 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.256455:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371126528:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302770.256664:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ef00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302770.267472:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.267631:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350364672 00000100:00100000:3.0:1789302770.267641:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350364672 00010000:00010000:3.0:1789302770.267679:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 103 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302770.267730:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 322 00000100:00100000:3.0:1789302770.267740:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f096b0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2348:x1876219350364672:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302770.267753:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302770.267760:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 103 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.267773:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 103 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.267794:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:3.0:1789302770.267800:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.267809:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.267817:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.267836:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302770.267926:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f096b0850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2348:x1876219350364672:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 187us (463us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.267934:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 322 00010000:00010000:1.0:1789302770.269852:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095184df4b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x40:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0659 expref: 102 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.269882:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371126528:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302770.269906:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.269928:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.269935:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270003:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270016:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270025:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270096:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270104:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270128:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270134:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302770.270139:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270146:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270158:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:1.0:1789302770.270160:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302770.270165:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270436:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e032 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x40:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270457:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270463:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270469:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270474:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302770.270478:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270494:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270550:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x60680aa5 (ffff960f10d24100) 00010000:00010000:1.0:1789302770.270554:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302770.270557:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.270563:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e039 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x40:0x0].0x60680aa5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.270709:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28076:x1876219350364544:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 14704us (21551us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302770.270719:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1066 00000100:00100000:0.0:1789302770.276442:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.276679:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350364800 00000100:00100000:3.0:1789302770.276690:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350364800 00000100:00100000:3.0:1789302770.276749:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1067 00000100:00100000:3.0:1789302770.276759:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28076:x1876219350364800:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302770.276780:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302770.276805:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e040 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06a6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.276840:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.276853:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.276865:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.276973:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e040 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.276993:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.277001:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302770.277004:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302770.277008:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1400, rc 0) 00000100:00100000:3.0:1789302770.277079:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28076:x1876219350364800:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 321us (646us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.277086:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1067 00000100:00100000:2.0:1789302770.285228:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.285471:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350365056 00000100:00100000:1.0:1789302770.285481:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350365056 00000100:00100000:1.0:1789302770.285567:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1068 00000100:00100000:1.0:1789302770.285576:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28076:x1876219350365056:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302770.285667:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.285687:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.285700:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302770.285914:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000032cd3abe reply state 0000000029aa0bff transno 4294967990 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.286764:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302770.286931:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:28076:x1876219350365056:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1353us (1708us total) trans 4294967990 rc 0/0 00000100:00100000:1.0:1789302770.286939:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1068 00000100:00100000:2.0:1789302770.292923:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.293181:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350365184 00000100:00100000:1.0:1789302770.293191:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350365184 00000100:00100000:1.0:1789302770.293270:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1069 00000100:00100000:1.0:1789302770.293279:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:28076:x1876219350365184:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302770.293302:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302770.293321:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06ad expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.293385:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.293399:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.293408:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302770.293411:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302770.293415:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6a00, rc 0) 00000100:00100000:1.0:1789302770.293467:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:28076:x1876219350365184:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 189us (563us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.293473:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1069 00010000:00010000:3.0:1789302770.295027:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.295042:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.295051:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.295058:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302770.295064:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.295073:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.295088:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302770.295092:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789302770.295097:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.295103:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e04e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.307266:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.307692:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350365312 00000100:00100000:3.0:1789302770.307702:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350365312 00000100:00100000:3.0:1789302770.307763:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 211 00000100:00100000:3.0:1789302770.307774:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ed450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:28076:x1876219350365312:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1789302770.307922:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ed450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:28076:x1876219350365312:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 149us (680us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.307929:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 211 00010000:00010000:2.0:1789302770.316762:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302770.318990:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.319157:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350365568 00000100:00100000:0.0:1789302770.319169:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350365568 00000100:00100000:0.0:1789302770.319258:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 212 00000100:00100000:0.0:1789302770.319278:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:28076:x1876219350365568:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302770.320035:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:28076:x1876219350365568:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 754us (1047us total) trans 4294967991 rc 0/0 00000100:00100000:2.0:1789302770.320044:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 212 00000100:00100000:0.0:1789302770.324147:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.324710:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350365824 00000100:00100000:1.0:1789302770.324720:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350365824 00000100:00100000:1.0:1789302770.324795:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1070 00000100:00100000:1.0:1789302770.324807:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:28076:x1876219350365824:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302770.325089:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:28076:x1876219350365824:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 283us (948us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.325097:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1070 00000100:00100000:2.0:1789302770.331580:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.331651:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350366080 00000100:00100000:1.0:1789302770.331662:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350366080 00000100:00100000:1.0:1789302770.331724:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1071 00000100:00100000:1.0:1789302770.331735:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:28076:x1876219350366080:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302770.331757:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302770.331777:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e05c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06b4 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.331886:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.331901:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 54 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.331909:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.331956:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 54 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.331970:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 54 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.332006:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371126656:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302770.333600:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f02cd9900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302770.342662:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.342987:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350366208 00000100:00100000:0.0:1789302770.342997:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350366208 00010000:00010000:0.0:1789302770.343035:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 55 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.343072:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 323 00000100:00100000:0.0:1789302770.343082:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2348:x1876219350366208:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302770.343095:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302770.343102:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 55 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.343127:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 55 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.343149:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302770.343154:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.343164:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.343184:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.343205:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302770.343275:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2348:x1876219350366208:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 194us (622us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.343283:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 323 00010000:00010000:1.0:1789302770.345397:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e055 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ad expref: 54 pid: 18427 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.345427:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267e680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371126656:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302770.345455:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.345466:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.345472:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302770.345495:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 0 00000004:00080000:1.0:1789302770.345837:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x16:0x0], index 1: 0 00010000:00010000:1.0:1789302770.346140:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346149:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346156:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346162:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302770.346167:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346324:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346402:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302770.346405:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302770.346411:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346417:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e063 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346472:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e05c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06b4 expref: 54 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346483:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x40:0x0].0x0 (ffff960f03501d00) 00010000:00010000:1.0:1789302770.346485:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302770.346489:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e05c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x40:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06b4 expref: 54 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.346498:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc 0) 00000100:00100000:1.0:1789302770.346783:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:28076:x1876219350366080:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 15048us (15210us total) trans 4294967992 rc 301/301 00000100:00100000:1.0:1789302770.346792:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1071 00000100:00100000:2.0:1789302770.351493:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.351719:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350366464 00000100:00100000:3.0:1789302770.351729:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350366464 00000100:00100000:2.0:1789302770.351885:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 213 00000100:00100000:2.0:1789302770.351895:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:28076:x1876219350366464:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302770.352766:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:28076:x1876219350366464:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 870us (1279us total) trans 4294967993 rc 0/0 00000100:00100000:2.0:1789302770.352776:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 213 00000100:00100000:1.0:1789302770.356492:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.356641:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350366592 00000100:00100000:0.0:1789302770.356651:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350366592 00000100:00100000:0.0:1789302770.356736:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1072 00000100:00100000:0.0:1789302770.356747:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:28076:x1876219350366592:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302770.356770:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302770.356793:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e071 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06bb expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.356860:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e078 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.356894:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e078 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.356905:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e078 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302770.356962:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x1e:0x0]: returning size 0 00000004:00200000:0.0:1789302770.356966:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302770.356985:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e078 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.357010:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e071 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06bb expref: 56 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.357028:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e078 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06bb expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.357036:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302770.357038:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1789302770.357042:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302770.357102:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:28076:x1876219350366592:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 357us (617us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.357108:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1072 00000100:00100000:3.0:1789302770.360746:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302770.360932:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350366720 00000100:00100000:2.0:1789302770.360953:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350366720 00000100:00100000:2.0:1789302770.361030:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 285 00000100:00100000:2.0:1789302770.361041:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350366720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302770.361058:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302770.361080:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06c2 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302770.361107:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x16:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302770.361128:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=607s v=(0 0) 00010000:00010000:2.0:1789302770.361133:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302770.361143:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.361199:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.361213:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00080000:3.0:1789302770.361526:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302770.361603:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350366720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 563us (866us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302770.361612:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 285 00000100:00100000:2.0:1789302770.366024:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.366206:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350366976 00000100:00100000:3.0:1789302770.366217:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350366976 00000100:00100000:3.0:1789302770.366292:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1073 00000100:00100000:3.0:1789302770.366304:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:28076:x1876219350366976:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302770.366678:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:28076:x1876219350366976:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 376us (663us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.366686:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1073 00000100:00100000:3.0:1789302770.370817:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.370936:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350367104 00000100:00100000:0.0:1789302770.371321:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350367104 00000100:00100000:0.0:1789302770.371543:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 286 00000100:00100000:0.0:1789302770.371556:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:28076:x1876219350367104:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302770.371575:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302770.371610:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06c9 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.371641:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.371650:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302770.371655:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302770.371659:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.371693:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.371707:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 11 pid: 6695 timeout: 707 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.371722:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371126784:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302770.371797:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ee40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302770.376203:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52a00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371126784:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302770.376245:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.376283:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:3.0:1789302770.376350:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:28076:x1876219350367104:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4796us (5541us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.376359:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 286 00000100:00100000:2.0:1789302770.380867:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.381069:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350367232 00000100:00100000:3.0:1789302770.381079:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350367232 00010000:00010000:3.0:1789302770.381147:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 11 pid: 6695 timeout: 707 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.382534:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 324 00000100:00100000:1.0:1789302770.382553:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350367232:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302770.382571:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302770.382607:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 11 pid: 6695 timeout: 707 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.382623:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 11 pid: 6695 timeout: 707 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.382647:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184e07f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c2 expref: 11 pid: 6695 timeout: 707 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.382663:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302770.382667:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302770.382674:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.382685:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.382724:0:8463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.382739:0:8463:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8463:1876219371126912:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302770.382827:0:8463:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdb180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302770.389831:0:8463:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0001_UUID:8463:1876219371126912:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302770.389859:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302770.389910:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350367232:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 7365us (9052us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.389934:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 324 00000100:00100000:0.0:1789302770.397368:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302770.398150:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350367360 00000100:00100000:2.0:1789302770.398160:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350367360 00002000:00100000:2.0:1789302770.398223:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x16:0x0] (0->1515) req@ffff960f3ef52a00 x1876219350367360/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:346/0 lens 2008/0 e 0 to 0 dl 1789302781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302770.398235:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef52a00 with x1876219350367360 ext(0->1515) 00002000:00010000:2.0:1789302770.398242:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ef52a00 00000100:00100000:2.0:1789302770.398483:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 99 00000100:00100000:2.0:1789302770.398494:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350367360:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302770.399559:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350367360:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 1064us (2197us total) trans 4294967357 rc 0/0 00000100:00100000:2.0:1789302770.399569:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 99 00002000:00100000:2.0:1789302770.399577:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x16:0x0] (0->1515) req@ffff960f3ef52a00 x1876219350367360/t4294967357(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:346/0 lens 2008/448 e 0 to 0 dl 1789302781 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302770.399586:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef52a00 with x1876219350367360 ext(0->1515) 00002000:00010000:2.0:1789302770.399598:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ef52a00 00000100:00100000:2.0:1789302770.406663:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.406886:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350367488 00000100:00100000:1.0:1789302770.406896:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350367488 00000100:00100000:1.0:1789302770.407109:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1074 00000100:00100000:1.0:1789302770.407121:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:28076:x1876219350367488:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1789302770.493158:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967986 is committed 00000004:00080000:3.0:1789302770.493171:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 14 committed 00000020:00080000:3.0:1789302770.493198:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f07e4e0c0 refcount 2 committed 0 result 0 batchid 19 00000020:00080000:3.0:1789302770.493203:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3b0b4400 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f07e4e480 00000020:00080000:3.0:1789302770.493206:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3b0b4b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f098ac400 00000001:00080000:3.0:1789302770.493224:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967975 is committed 00000001:00080000:3.0:1789302770.493230:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967992 is committed 00000001:00080000:3.0:1789302770.493284:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967993 is committed 00000001:00080000:3.0:1789302770.493302:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294967987 is committed 00000020:00080000:3.0:1789302770.493321:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 19 committed batchid 18 00000020:00080000:3.0:1789302770.493555:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 18 00000100:00100000:1.0:1789302770.493834:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28076:x1876219350367488:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 86711us (87178us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.493845:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1074 00000100:00100000:1.0:1789302770.494048:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f709180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371127040:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789302770.494099:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302770.494413:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371127040 00000100:00100000:3.0:1789302770.494421:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371127040 00000100:00100000:3.0:1789302770.494477:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789302770.494487:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371127040:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1789302770.494534:0:8571:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:22 count 1 00002000:00010000:3.0:1789302770.494564:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x16:0x0] initial LVB size: 1516, mtime: 0x6aa697f0, atime: 0x0, ctime: 0x6aa697f0, blocks: 0x1 00010000:00010000:3.0:1789302770.494573:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 3/0,1 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.494603:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=607s v=(0 0) 00010000:00010000:3.0:1789302770.494607:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302770.494614:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.494628:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.494637:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.494646:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.494663:0:8571:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789302770.495779:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302770.495789:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.495799:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302770.495803:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.495812:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.495828:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 1/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.495835:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d13f855e/0x64fe43095184e08d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.495854:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302770.495856:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302770.495858:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302770.496265:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+7:3303:x1876219371127040:12345-0@lo:6:osp-syn-0-0.0 Request processed in 1769us (2154us total) trans 4294967353 rc 0/0 00000100:00100000:3.0:1789302770.496276:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:2.0:1789302770.498495:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.499307:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350367616 00000100:00100000:3.0:1789302770.499317:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350367616 00000100:00100000:3.0:1789302770.499388:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:3.0:1789302770.499398:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350367616:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000004:00080000:1.0:1789302770.504787:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f709180/2, rc 0, transno 4294967353 00000100:00100000:1.0:1789302770.504800:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f709180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371127040:0@lo:6:osp-syn-0-0.0 00000001:00080000:2.0:1789302770.559899:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967357 is committed 00000100:00100000:1.0:1789302770.560109:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350367616:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 60707us (61615us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.560120:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:0.0:1789302770.575790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.577497:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350367872 00000100:00100000:3.0:1789302770.577532:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350367872 00000100:00100000:3.0:1789302770.577605:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1075 00000100:00100000:3.0:1789302770.577614:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28076:x1876219350367872:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302770.577684:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302770.577710:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e078 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06bb expref: 44 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.577731:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302770.577736:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302770.577768:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.577781:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.577791:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302770.577981:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b3d07bf9 reply state 0000000053d57c3a transno 4294967994 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302770.577990:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302770.578104:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28076:x1876219350367872:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 491us (2324us total) trans 4294967994 rc 0/0 00000100:00100000:3.0:1789302770.578112:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1075 00000100:00100000:1.0:1789302770.583701:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302770.584168:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.584182:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.584220:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.584227:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302770.584233:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.584241:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.584268:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:2.0:1789302770.584271:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302770.584276:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.584283:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e094 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302770.584441:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350368000 00000100:00100000:3.0:1789302770.584453:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350368000 00000100:00100000:3.0:1789302770.584543:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1076 00000100:00100000:3.0:1789302770.584554:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28076:x1876219350368000:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302770.584577:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302770.584596:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06d0 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.584674:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.584687:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302770.584706:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302770.584709:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302770.584712:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302770.584777:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28076:x1876219350368000:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 225us (1085us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.584784:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1076 00000100:00100000:2.0:1789302770.591422:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.591538:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350368128 00000100:00100000:1.0:1789302770.591549:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350368128 00000100:00100000:1.0:1789302770.591613:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 289 00000100:00100000:1.0:1789302770.591623:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350368128:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1789302770.591888:0:6696:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb size from disk: 0 -> 1516 00002000:00010000:1.0:1789302770.591892:0:6696:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb mtime from disk: 0 -> 1789302768 00002000:00010000:1.0:1789302770.591894:0:6696:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb atime from disk: 0 -> 1789302768 00002000:00010000:1.0:1789302770.591896:0:6696:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb ctime from disk: 0 -> 1789302768 00002000:00010000:1.0:1789302770.591899:0:6696:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x16:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1789302770.591977:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350368128:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 355us (565us total) trans 4294967358 rc 0/0 00000100:00100000:1.0:1789302770.591986:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 289 00000100:00100000:0.0:1789302770.599652:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.599814:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350368384 00000100:00100000:1.0:1789302770.599835:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350368384 00000100:00100000:1.0:1789302770.599901:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1077 00000100:00100000:1.0:1789302770.599912:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28076:x1876219350368384:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1789302770.599977:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.599995:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.600003:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.600037:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.600051:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 45 pid: 5845 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.600065:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371127168:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302770.600163:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdb180 going to sleep for 11 seconds 00000100:00100000:2.0:1789302770.611141:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.611243:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350368512 00000100:00100000:0.0:1789302770.611253:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350368512 00010000:00010000:0.0:1789302770.611306:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 46 pid: 5845 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.611374:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:0.0:1789302770.611384:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350368512:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302770.611399:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302770.611406:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 46 pid: 5845 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.611418:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 46 pid: 5845 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.611440:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302770.611445:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.611455:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.611464:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.611484:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302770.611575:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350368512:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 192us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.611582:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 325 00010000:00010000:1.0:1789302770.613396:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e047 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06a6 expref: 45 pid: 5845 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.613420:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371127168:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302770.613444:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.613454:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.613480:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.617525:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.617543:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.617551:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.617557:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302770.617562:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.617571:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.617589:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302770.617592:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1789302770.617597:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.617603:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0a2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.617734:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28076:x1876219350368384:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 17818us (18097us total) trans 0 rc -22/-22 00000100:00100000:1.0:1789302770.617745:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1077 00000100:00100000:2.0:1789302770.621483:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.621615:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350368640 00000100:00100000:0.0:1789302770.621625:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350368640 00000100:00100000:0.0:1789302770.621699:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 290 00000100:00100000:0.0:1789302770.621710:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350368640:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302770.621774:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.621795:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.621804:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302770.621809:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302770.621814:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.621849:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.621862:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 11 pid: 18578 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.621876:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371127296:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302770.621960:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea000 going to sleep for 11 seconds 00000100:00100000:0.0:1789302770.624675:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371127296:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302770.624708:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302770.627965:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.628073:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350368768 00000100:00100000:1.0:1789302770.628082:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350368768 00010000:00010000:1.0:1789302770.628124:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 12 pid: 18578 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.628162:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:1.0:1789302770.628172:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350368768:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302770.628187:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302770.628210:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 12 pid: 18578 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.628235:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 12 pid: 18578 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.628267:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e086 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x16:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06c9 expref: 12 pid: 18578 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.628285:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302770.628289:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302770.628294:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.628303:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.628337:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302770.628392:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350368768:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 221us (436us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.628415:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 326 00010000:00010000:0.0:1789302770.628777:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628800:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628825:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628833:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628842:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628848:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302770.628852:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628860:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628872:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.628880:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000067e04838/0x64fe43095184e0a9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.628954:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350368640:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 7245us (7479us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.628963:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 290 00000100:00100000:2.0:1789302770.633838:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302770.633949:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350369024 00000100:00100000:1.0:1789302770.633959:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350369024 00000100:00100000:1.0:1789302770.634017:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1078 00000100:00100000:1.0:1789302770.634028:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28076:x1876219350369024:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1789302770.634099:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.634116:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.634127:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.634363:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.634375:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.634385:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.634417:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.634430:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 45 pid: 5844 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.634465:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371127424:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302770.634574:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdb180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302770.640111:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.640234:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350369152 00000100:00100000:0.0:1789302770.640245:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350369152 00010000:00010000:0.0:1789302770.640293:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 46 pid: 5844 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302770.640350:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 327 00000100:00100000:0.0:1789302770.640361:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350369152:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302770.640376:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302770.640383:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 46 pid: 5844 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.640395:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 46 pid: 5844 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.640417:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302770.640423:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.640433:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.640440:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.640490:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302770.640574:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350369152:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 214us (471us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.640582:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 327 00010000:00010000:1.0:1789302770.641302:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e09b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d0 expref: 45 pid: 5844 timeout: 707 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302770.641322:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371127424:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302770.642557:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.642575:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.642581:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789302770.642786:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967995 00000001:00080000:1.0:1789302770.642941:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967996 00000001:00080000:1.0:1789302770.642985:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967997 00000100:00100000:1.0:1789302770.643062:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371127552:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789302770.643143:0:18427:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302770.643188:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f02cd9900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302770.645412:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371127552 00000100:00100000:2.0:1789302770.645429:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371127552 00000100:00100000:2.0:1789302770.645563:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 43 00000100:00100000:2.0:1789302770.645574:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:18427:x1876219371127552:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789302770.645702:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:18427:x1876219371127552:12345-0@lo:1000:mdt00_004.0 Request processed in 130us (2558us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302770.645726:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 43 00000100:00100000:2.0:1789302770.645781:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371127552:0@lo:1000:mdt00_004.0 00000001:00080000:2.0:1789302770.646166:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967998 00000100:00100000:2.0:1789302770.646398:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbc380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371127680:0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789302770.646440:0:18427:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302770.646469:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302770.646489:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371127680 00000100:00100000:2.0:1789302770.646493:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371127680 00000100:00100000:2.0:1789302770.646537:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789302770.646544:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:18427:x1876219371127680:12345-0@lo:1000:mdt00_004.0 00000100:00100000:2.0:1789302770.646708:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:18427:x1876219371127680:12345-0@lo:1000:mdt00_004.0 Request processed in 164us (267us total) trans 4294967359 rc 0/0 00000100:00100000:2.0:1789302770.646715:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 44 00000100:00100000:2.0:1789302770.646748:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbc380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371127680:0@lo:1000:mdt00_004.0 00000001:00080000:2.0:1789302770.646824:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294967999 00010000:00010000:2.0:1789302770.646850:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646860:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646868:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646873:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302770.646877:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646896:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646911:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:2.0:1789302770.646914:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302770.646920:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646926:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e0b7 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1e:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646956:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646962:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646968:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646973:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302770.646976:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646981:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.646990:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927700) 00010000:00010000:2.0:1789302770.646992:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302770.647000:0:18427:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0691 expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.647023:0:18427:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0691 expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.647032:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0691 expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.647058:0:18427:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1600/0x64fe43095184dfec lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0691 expref: 90 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302770.647069:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbc700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371127808:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302770.647128:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a900 going to sleep for 11 seconds 00000100:00080000:3.0:1789302770.650971:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302770.652724:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbc700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371127808:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302770.652755:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652765:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184dfe5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652799:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652813:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652823:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652833:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652853:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652859:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652864:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302770.652867:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652874:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652884:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927700) 00010000:00010000:2.0:1789302770.652887:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302770.652891:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652896:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e0be lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652905:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652910:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652915:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652920:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302770.652929:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652936:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652943:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1d:0x0].0x0 (ffff960f03927700) 00010000:00010000:2.0:1789302770.652946:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302770.652949:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302770.652955:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e0b0 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302770.653387:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302770.653594:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28076:x1876219350369024:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 19567us (19764us total) trans 4294968000 rc 0/0 00000100:00100000:2.0:1789302770.653605:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1078 00000020:00080000:2.0:1789302770.653612:0:18427:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:1.0:1789302770.654810:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302770.654817:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:0.0:1789302770.655848:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350369280 00000100:00100000:0.0:1789302770.655859:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350369280 00000100:00100000:0.0:1789302770.655987:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 214 00000100:00100000:0.0:1789302770.655998:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28075:x1876219350369280:12345-192.168.203.11@tcp:49:cmp.0 00000100:00100000:0.0:1789302770.656135:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28075:x1876219350369280:12345-192.168.203.11@tcp:49:cmp.0 Request processed in 139us (2750us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.656143:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 214 00000040:00080000:1.0:1789302770.656276:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1789302770.656287:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302770.667081:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.667122:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.667250:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350369536 00000100:00100000:0.0:1789302770.667268:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350369536 00000100:00100000:0.0:1789302770.667323:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 215 00000100:00100000:0.0:1789302770.667335:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28076:x1876219350369536:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:0.0:1789302770.667987:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:0.0:1789302770.668090:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968001 00010000:00010000:0.0:1789302770.668132:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668147:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668158:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668166:0:5846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668179:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668185:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668208:0:5846:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302770.668307:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:28076:x1876219350369536:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 975us (1233us total) trans 4294968002 rc 0/0 00000100:00100000:0.0:1789302770.668316:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 215 00000020:00080000:0.0:1789302770.668322:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1789302770.668350:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302770.668356:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668363:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302770.668367:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668375:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668387:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1e:0x0].0x0 (ffff960f03501800) 00010000:00010000:0.0:1789302770.668390:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302770.668395:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668400:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e0c5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302770.668428:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302770.668429:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302770.668431:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302770.675238:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350369664 00000100:00100000:1.0:1789302770.675253:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350369664 00000100:00080000:3.0:1789302770.675455:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302770.675533:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 291 00000100:00100000:1.0:1789302770.675544:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:28075:x1876219350369664:12345-192.168.203.11@tcp:101:cmp.0 00010000:00010000:1.0:1789302770.675563:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302770.675582:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f36d5065/0x64fe43095184e0cc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06d7 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302770.675608:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x16:0x0] initial LVB size: 1516, mtime: 0x6aa697f0, atime: 0x6aa697f0, ctime: 0x6aa697f0, blocks: 0xb 00010000:00010000:1.0:1789302770.675628:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=607s v=(0 0) 00010000:00010000:1.0:1789302770.675633:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x16:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302770.676006:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f36d5065/0x64fe43095184e0cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d7 expref: 16 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.676025:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f36d5065/0x64fe43095184e0cc lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x16:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06d7 expref: 16 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302770.676037:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6000, rc 0) 00000100:00100000:1.0:1789302770.676106:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:28075:x1876219350369664:12345-192.168.203.11@tcp:101:cmp.0 Request processed in 564us (8982us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.676114:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 291 00000100:00100000:3.0:1789302770.679313:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302770.679474:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350369792 00000100:00100000:2.0:1789302770.679484:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350369792 00002000:00100000:2.0:1789302770.679572:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x16:0x0] (0->4095) req@ffff960e011c7100 x1876219350369792/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:346/0 lens 488/0 e 0 to 0 dl 1789302781 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302770.679583:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c7100 with x1876219350369792 ext(0->4095) 00002000:00010000:2.0:1789302770.679589:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e011c7100 00000100:00100000:2.0:1789302770.679614:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 100 00000100:00100000:2.0:1789302770.679623:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350369792:12345-192.168.203.11@tcp:3:cmp.0 00000100:00100000:2.0:1789302770.679817:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350369792:12345-192.168.203.11@tcp:3:cmp.0 Request processed in 195us (514us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1789302770.679825:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 100 00002000:00100000:2.0:1789302770.679845:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x16:0x0] (0->4095) req@ffff960e011c7100 x1876219350369792/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:346/0 lens 488/4536 e 0 to 0 dl 1789302781 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302770.679854:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e011c7100 with x1876219350369792 ext(0->4095) 00002000:00010000:2.0:1789302770.679857:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e011c7100 00000100:00100000:2.0:1789302770.688924:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302770.689066:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370048 00000100:00100000:0.0:1789302770.689077:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370048 00000100:00100000:0.0:1789302770.689142:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 216 00000100:00100000:0.0:1789302770.689152:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28075:x1876219350370048:12345-192.168.203.11@tcp:35:cmp.0 00000100:00100000:0.0:1789302770.689409:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28075:x1876219350370048:12345-192.168.203.11@tcp:35:cmp.0 Request processed in 257us (782us total) trans 4294968003 rc 0/0 00000100:00100000:0.0:1789302770.689417:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 216 00000100:00100000:1.0:1789302770.701749:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371128320:0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.701817:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302770.701886:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789302770.701893:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371128192:0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.701918:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302770.701991:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371127936:0@lo:400:kworker.0 00000100:00100000:2.0:1789302770.702025:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302770.702056:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302770.702062:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371128064:0@lo:400:kworker.0 00000100:00100000:2.0:1789302770.702085:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302770.702153:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371127936 00000100:00100000:1.0:1789302770.702161:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371127936 00000100:00100000:1.0:1789302770.702238:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00100000:1.0:1789302770.702248:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371127936:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.702310:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371127936:12345-0@lo:400:kworker.0 Request processed in 64us (283us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.702317:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00100000:1.0:1789302770.702363:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371127936:0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.702385:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128320 00000100:00100000:1.0:1789302770.702389:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128320 00000100:00100000:1.0:1789302770.702421:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128192 00000100:00100000:1.0:1789302770.702424:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128192 00000100:00100000:1.0:1789302770.702442:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128064 00000100:00100000:1.0:1789302770.702444:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128064 00000100:00100000:1.0:1789302770.702457:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789302770.702463:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371128320:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.702498:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371128320:12345-0@lo:400:kworker.0 Request processed in 35us (681us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.702539:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000100:00100000:1.0:1789302770.702552:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789302770.702558:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371128192:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.702589:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371128192:12345-0@lo:400:kworker.0 Request processed in 32us (671us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.702595:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:1.0:1789302770.702607:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:1.0:1789302770.702613:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371128064:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.702767:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371128064:12345-0@lo:400:kworker.0 Request processed in 153us (680us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302770.702784:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00080000:1.0:1789302770.702802:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302770.702838:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371128064:0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.702864:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371128192:0@lo:400:kworker.0 00000100:00100000:1.0:1789302770.702872:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371128320:0@lo:400:kworker.0 00000100:00100000:3.0:1789302770.960987:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.961038:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.961075:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.961114:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.961142:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.961170:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302770.961199:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370432 00000100:00100000:3.0:1789302770.961223:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370432 00000100:00100000:3.0:1789302770.961282:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370688 00000100:00100000:3.0:1789302770.961286:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370688 00000100:00100000:3.0:1789302770.961309:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370304 00000100:00100000:3.0:1789302770.961312:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370304 00000100:00100000:3.0:1789302770.961324:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370816 00000100:00100000:3.0:1789302770.961327:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370816 00000100:00100000:3.0:1789302770.961349:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 391 00000100:00100000:3.0:1789302770.961359:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2345:x1876219350370432:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302770.961427:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2345:x1876219350370432:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 70us (388us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.961434:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 391 00000100:00100000:3.0:1789302770.961461:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 392 00000100:00100000:3.0:1789302770.961468:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350370688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302770.961494:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350370688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (419us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.961525:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 392 00000100:00100000:2.0:1789302770.961537:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 393 00000100:00100000:2.0:1789302770.961546:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350370304:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302770.961578:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350370304:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (464us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.961588:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370176 00000100:00080000:1.0:1789302770.961631:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302770.961718:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370176 00000100:00100000:0.0:1789302770.961751:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 563 00000100:00100000:0.0:1789302770.961758:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350370176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302770.961787:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350370176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (810us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302770.961793:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 563 00000100:00100000:3.0:1789302770.962403:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 394 00000100:00100000:3.0:1789302770.962414:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350370816:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302770.962457:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859b450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350370816:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (1275us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.962463:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 394 00000100:00100000:3.0:1789302770.966716:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370560 00000100:00100000:3.0:1789302770.966733:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370560 00000100:00100000:3.0:1789302770.966803:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 400 00000100:00100000:3.0:1789302770.966814:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:2345:x1876219350370560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302770.966907:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:2345:x1876219350370560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 95us (5763us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302770.966915:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 400 00000100:00100000:2.0:1789302770.967371:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 393 00000100:00100000:2.0:1789302771.213534:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371128448:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302771.213600:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302771.213888:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128448 00000100:00100000:0.0:1789302771.213896:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128448 00000100:00100000:0.0:1789302771.213945:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:0.0:1789302771.213954:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371128448:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302771.213977:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 78184448 pending 0 free 3856662528 avail 3853516800 00000100:00100000:0.0:1789302771.214023:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371128448:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302771.214030:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:1.0:1789302771.214155:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371128448:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302771.343073:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302771.725324:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302771.725347:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371128576:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302771.725421:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302771.725538:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128576 00000100:00100000:2.0:1789302771.725547:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128576 00000100:00100000:2.0:1789302771.725597:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 173 00000100:00100000:2.0:1789302771.725607:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3300:x1876219371128576:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302771.725654:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 79888384 pending 1703936 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302771.725705:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3300:x1876219371128576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 100us (284us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302771.725733:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 173 00000100:00100000:2.0:1789302771.725781:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371128576:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302772.366429:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302773.339914:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967352 is committed 00000001:00080000:2.0:1789302773.339942:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967351 is committed 00000001:00080000:2.0:1789302773.339950:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967353 is committed 00010000:00010000:0.0:1789302773.388775:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302774.414276:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302775.439411:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302775.823530:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371128704:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.823765:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302775.823813:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371129216:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.823841:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302775.823870:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789302775.823876:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371128832:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.823897:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302775.823915:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371129344:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.823942:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302775.823976:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371128960:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.824003:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302775.824022:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302775.824027:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371129088:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.824053:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302775.824055:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371129216 00000100:00100000:3.0:1789302775.824065:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371129216 00000100:00100000:0.0:1789302775.824088:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128704 00000100:00100000:0.0:1789302775.824094:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128704 00000100:00100000:3.0:1789302775.825213:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128832 00000100:00100000:3.0:1789302775.825230:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128832 00000100:00100000:3.0:1789302775.825265:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371128960 00000100:00100000:3.0:1789302775.825268:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371128960 00000100:00100000:3.0:1789302775.825288:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:3.0:1789302775.825308:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371129216:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.825311:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 564 00000100:00100000:0.0:1789302775.825326:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371128704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.825392:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371128704:12345-0@lo:400:kworker.0 Request processed in 70us (1627us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302775.825399:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 564 00000100:00100000:0.0:1789302775.825437:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1789302775.825444:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371128832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302775.825729:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371129216:12345-0@lo:400:kworker.0 Request processed in 432us (1884us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302775.825736:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:3.0:1789302775.825754:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789302775.825761:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371128960:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302775.825799:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371128960:12345-0@lo:400:kworker.0 Request processed in 39us (1795us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302775.825805:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1789302775.825822:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371129344 00000100:00100000:3.0:1789302775.825826:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371129344 00000100:00100000:3.0:1789302775.825852:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371129088 00000100:00100000:3.0:1789302775.825854:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371129088 00000100:00100000:3.0:1789302775.825871:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789302775.825947:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371129344:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302775.825990:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371129344:12345-0@lo:400:kworker.0 Request processed in 46us (2047us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302775.825996:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:3.0:1789302775.826009:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:3.0:1789302775.826016:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371129088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.826038:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371128832:12345-0@lo:400:kworker.0 Request processed in 593us (2138us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302775.826051:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371129088:12345-0@lo:400:kworker.0 Request processed in 34us (1997us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302775.826058:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:3.0:1789302775.826062:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00080000:3.0:1789302775.826090:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302775.826094:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302775.826097:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302775.828520:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371128832:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.828930:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371129344:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.828946:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371128704:0@lo:400:kworker.0 00000100:00100000:0.0:1789302775.828958:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371129216:0@lo:400:kworker.0 00000100:00100000:3.0:1789302775.832256:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967353 gen 2 00000004:00080000:3.0:1789302775.832270:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967352 committed transno 4294967353 00000004:00080000:3.0:1789302775.832296:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f709180, transno 4294967353 00000100:00100000:3.0:1789302775.832315:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371129088:0@lo:400:kworker.0 00000100:00100000:3.0:1789302775.832407:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371128960:0@lo:400:kworker.0 00000001:00080000:3.0:1789302775.887123:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968003 is committed 00000001:00080000:3.0:1789302775.887135:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968002 is committed 00000020:00080000:3.0:1789302775.887162:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f3f73aa80 refcount 2 committed 0 result 0 batchid 20 00000020:00080000:3.0:1789302775.887168:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960e02228b00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f3f73a000 00000020:00080000:3.0:1789302775.887171:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960e02228800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960e044c3400 00000020:00080000:3.0:1789302775.887192:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 19 batchid updated 00000020:00080000:1.0:1789302775.887236:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 20 committed batchid 19 00000020:00080000:1.0:1789302775.887590:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 19 00000001:00080000:2.0:1789302775.890894:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967358 is committed 00000001:00080000:2.0:1789302775.890913:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967359 is committed 00000100:00100000:3.0:1789302775.901628:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302775.901728:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350370944 00000100:00100000:0.0:1789302775.901739:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350370944 00000100:00100000:0.0:1789302775.901812:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1079 00000100:00100000:0.0:1789302775.901822:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28056:x1876219350370944:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1789302775.901925:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28056:x1876219350370944:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 104us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302775.901932:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1079 00000100:00080000:3.0:1789302775.902528:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302776.086394:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302776.086445:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302776.086561:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302776.086582:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302776.086608:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302776.086636:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371200 00000100:00100000:3.0:1789302776.086645:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371200 00000100:00100000:0.0:1789302776.086889:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371328 00000100:00100000:0.0:1789302776.086894:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371328 00000100:00100000:0.0:1789302776.086932:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371584 00000100:00100000:0.0:1789302776.086937:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371584 00000100:00100000:1.0:1789302776.086956:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371072 00000100:00100000:0.0:1789302776.086962:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371456 00000100:00100000:1.0:1789302776.086963:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371072 00000100:00100000:0.0:1789302776.086967:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371456 00000100:00100000:0.0:1789302776.086999:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 397 00000100:00100000:1.0:1789302776.087012:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 565 00000100:00100000:0.0:1789302776.087014:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350371328:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302776.087026:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350371072:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302776.087082:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350371072:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 61us (697us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302776.087091:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 565 00000100:00100000:2.0:1789302776.087112:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 398 00000100:00100000:2.0:1789302776.087124:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350371584:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302776.087154:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 399 00000100:00100000:1.0:1789302776.087163:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350371456:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302776.087186:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:3.0:1789302776.087199:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2347:x1876219350371200:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302776.088557:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350371328:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1540us (2102us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302776.088569:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 397 00000100:00080000:0.0:1789302776.088625:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302776.088643:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302776.088646:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302776.090054:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350371584:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 2924us (3483us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302776.090069:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 398 00000100:00100000:0.0:1789302776.091359:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302776.093387:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371712 00000100:00100000:2.0:1789302776.093403:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371712 00000100:00100000:2.0:1789302776.093547:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 400 00000100:00100000:2.0:1789302776.093559:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350371712:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302776.093640:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350371712:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 85us (2287us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302776.093648:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 400 00000100:00080000:0.0:1789302776.095320:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302776.100773:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350371456:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 13599us (14150us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302776.100809:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 399 00000100:00100000:3.0:1789302776.101333:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2347:x1876219350371200:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 14132us (14745us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302776.101343:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:3.0:1789302776.332803:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371129472:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302776.332897:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302776.333067:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371129472 00000100:00100000:1.0:1789302776.333076:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371129472 00000100:00100000:1.0:1789302776.333131:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1789302776.333139:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371129472:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302776.333176:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 78184448 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302776.333241:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371129472:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (343us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302776.333248:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:3.0:1789302776.333394:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967359 gen 2 00000004:00080000:3.0:1789302776.333400:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967359 committed transno 4294967359 00000100:00100000:3.0:1789302776.333427:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371129472:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302776.460755:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302776.845855:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302776.845880:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371129600:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302776.845958:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302776.846209:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371129600 00000100:00100000:0.0:1789302776.846226:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371129600 00000100:00100000:0.0:1789302776.846279:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 175 00000100:00100000:0.0:1789302776.846297:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371129600:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302776.846335:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 3407872 granted 78184448 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302776.846393:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371129600:12345-0@lo:13:osp-pre-0-0.0 Request processed in 107us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302776.846400:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 175 00000100:00100000:1.0:1789302776.846618:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371129600:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302777.490036:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302777.639090:0:20233:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12n: Import/implicit restore/release == 08:32:55 (1789302775) 00000100:00100000:2.0:1789302778.012043:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.012142:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371840 00000100:00100000:0.0:1789302778.012152:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371840 00000100:00100000:0.0:1789302778.012242:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1080 00000100:00100000:0.0:1789302778.012253:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28680:x1876219350371840:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1789302778.012356:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28680:x1876219350371840:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 104us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302778.012364:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1080 00000100:00080000:3.0:1789302778.012485:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302778.039349:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.039407:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302778.039565:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350372224 00000100:00100000:1.0:1789302778.039578:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350372224 00000100:00100000:1.0:1789302778.039677:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350371968 00000100:00100000:1.0:1789302778.039681:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350371968 00000100:00100000:1.0:1789302778.039708:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1081 00000100:00100000:1.0:1789302778.039719:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28681:x1876219350372224:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:1.0:1789302778.039796:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302778.039835:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184de3a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a05f7 expref: 92 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.039859:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:1.0:1789302778.039865:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302778.040138:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.040162:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 41 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.040172:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.040224:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 41 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.040240:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 41 pid: 18427 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302778.040257:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371129728:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302778.040374:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73ab40 going to sleep for 11 seconds 00000100:00080000:3.0:1789302778.040839:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302778.040939:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1082 00000100:00100000:2.0:1789302778.040955:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2345:x1876219350371968:12345-192.168.203.11@tcp:41:mkdir.0 00000004:00000024:2.0:1789302778.041011:0:5843:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 14680064 pending 0 free 2259812352 avail 2257715200 00000100:00100000:2.0:1789302778.041072:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:2345:x1876219350371968:12345-192.168.203.11@tcp:41:mkdir.0 Request processed in 122us (1664us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302778.041080:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1082 00000100:00100000:1.0:1789302778.052317:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371129728:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302778.052364:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302778.052473:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302778.052607:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350372352 00000100:00100000:1.0:1789302778.052631:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350372352 00010000:00010000:1.0:1789302778.052679:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 42 pid: 18427 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302778.052721:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 328 00000100:00100000:1.0:1789302778.052732:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:2347:x1876219350372352:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302778.052752:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302778.052777:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 42 pid: 18427 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.052790:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 42 pid: 18427 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.052818:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184de79 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a060c expref: 42 pid: 18427 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.052838:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:1.0:1789302778.052844:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.052855:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.052863:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.053115:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302778.053182:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:2347:x1876219350372352:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 451us (710us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302778.053190:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 328 00010000:00010000:2.0:1789302778.053262:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.053271:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.053310:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.053320:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.053328:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302778.060188:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 16384 00010000:00010000:2.0:1789302778.060230:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.060243:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.060252:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.060259:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302778.060272:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.060292:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.060316:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:2.0:1789302778.060320:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302778.060327:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.060334:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e0d3 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302778.060360:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b0b6d5e3 reply state 00000000687d284f transno 4294968004 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302778.060370:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302778.060483:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28681:x1876219350372224:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 20760us (21143us total) trans 4294968004 rc 0/0 00000100:00100000:2.0:1789302778.060493:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1081 00010000:00010000:3.0:1789302778.081114:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.081139:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.081148:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.081155:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302778.081161:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.081192:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.081227:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff960f02d72700) 00010000:00010000:3.0:1789302778.081231:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302778.081238:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.081246:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0da lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.514601:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302778.583540:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302778.583711:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350372480 00000100:00100000:1.0:1789302778.583725:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350372480 00000100:00100000:1.0:1789302778.583808:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1083 00000100:00100000:1.0:1789302778.583820:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28718:x1876219350372480:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:1.0:1789302778.583869:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302778.583902:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e0e1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06de expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.584020:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e0e8 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.584038:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e0e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.584053:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e0e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302778.584104:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302778.584109:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302778.584132:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e0e8 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.584160:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e0e1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06de expref: 93 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.584181:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e0e8 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06de expref: 92 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.584190:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302778.584193:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302778.584198:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6800, rc 0) 00000100:00100000:1.0:1789302778.586548:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28718:x1876219350372480:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 2713us (3035us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302778.586563:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1083 00000100:00100000:2.0:1789302778.597701:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.597868:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350372736 00000100:00100000:0.0:1789302778.597882:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350372736 00000100:00100000:0.0:1789302778.597965:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1084 00000100:00100000:0.0:1789302778.597976:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28718:x1876219350372736:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:0.0:1789302778.598052:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302778.598102:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e0e8 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06de expref: 92 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.598125:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:0.0:1789302778.598137:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302778.598709:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28718:x1876219350372736:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 732us (1021us total) trans 0 rc -17/-17 00000100:00100000:0.0:1789302778.598719:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1084 00000100:00100000:3.0:1789302778.604824:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302778.604956:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350372864 00000100:00100000:2.0:1789302778.604969:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350372864 00000100:00100000:2.0:1789302778.605042:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1085 00000100:00100000:2.0:1789302778.605053:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+91:28718:x1876219350372864:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:2.0:1789302778.605080:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302778.605109:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e0ef lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06e5 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.605226:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0f6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.605243:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.605257:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302778.605450:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302778.605454:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302778.605478:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.605530:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e0ef lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06e5 expref: 93 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.605550:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0f6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06e5 expref: 92 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.605560:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:2.0:1789302778.605563:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302778.605567:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2200, rc 0) 00000100:00100000:2.0:1789302778.605645:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28718:x1876219350372864:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 594us (836us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302778.605653:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1085 00000100:00100000:1.0:1789302778.610048:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.610408:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350372992 00000100:00100000:0.0:1789302778.610421:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350372992 00000100:00100000:0.0:1789302778.610495:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1086 00000100:00100000:0.0:1789302778.610547:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:28718:x1876219350372992:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:0.0:1789302778.610576:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302778.610604:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e0fd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06ec expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610690:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610707:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610721:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610738:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610745:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610772:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610893:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e112 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610902:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e112 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.610910:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e112 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302778.612111:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 16384 00000004:00200000:0.0:1789302778.612121:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302778.612150:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e112 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612168:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612176:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612186:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612192:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302778.612197:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612215:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612245:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:0.0:1789302778.612249:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302778.612255:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612262:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e104 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612367:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612376:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612383:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612388:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302778.612391:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612397:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612406:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff960f03501000) 00010000:00010000:0.0:1789302778.612408:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302778.612411:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612418:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e10b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612473:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e0fd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ec expref: 94 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612490:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e112 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ec expref: 93 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.612498:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03501600) 00010000:00010000:0.0:1789302778.612517:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302778.612520:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:0.0:1789302778.612664:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28718:x1876219350372992:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 2119us (2628us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302778.612674:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1086 00000100:00100000:0.0:1789302778.700942:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302778.701135:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350373120 00000100:00100000:3.0:1789302778.701149:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350373120 00000100:00100000:3.0:1789302778.701231:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1087 00000100:00100000:3.0:1789302778.701245:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+41:28724:x1876219350373120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302778.701295:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302778.701329:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e119 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06f3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.701428:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.701448:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.701463:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302778.701603:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302778.701608:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302778.701637:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.701672:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e119 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 43 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.701698:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 42 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.701708:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:3.0:1789302778.701712:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302778.701717:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302778.701807:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28724:x1876219350373120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 566us (898us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302778.701814:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1087 00000100:00100000:3.0:1789302778.710444:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302778.710814:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350373376 00000100:00100000:2.0:1789302778.710828:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350373376 00000100:00100000:2.0:1789302778.710946:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1088 00000100:00100000:2.0:1789302778.710958:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28724:x1876219350373376:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302778.710989:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302778.710993:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:2.0:1789302778.711031:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e112 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06ec expref: 93 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.711056:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03501600) 00010000:00010000:2.0:1789302778.711071:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302778.711093:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e127 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a06fa expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.711628:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.711648:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.711661:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.711818:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.711844:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.711857:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712604:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712621:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712629:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302778.712641:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 0 00010000:00010000:2.0:1789302778.712666:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712673:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712680:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712686:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302778.712691:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712698:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712710:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f02d72700) 00010000:00010000:2.0:1789302778.712713:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302778.712718:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712725:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e12e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302778.712738:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000718f09fa reply state 00000000ec346034 transno 4294968005 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302778.712753:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302778.712792:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e127 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 95 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712810:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 94 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.712818:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302778.712820:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302778.712822:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302778.712900:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:28724:x1876219350373376:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1943us (2474us total) trans 4294968005 rc 0/0 00000100:00100000:2.0:1789302778.712909:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1088 00010000:00010000:2.0:1789302778.716398:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.716427:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.716436:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.716442:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302778.716447:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.716455:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.716494:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x81b47e8f (ffff960f03927400) 00010000:00010000:2.0:1789302778.716497:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302778.716522:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.716529:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e135 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302778.720285:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.720703:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350373632 00000100:00100000:0.0:1789302778.720718:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350373632 00000100:00100000:0.0:1789302778.720791:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1089 00000100:00100000:0.0:1789302778.720804:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:28724:x1876219350373632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302778.720841:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302778.720884:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e14a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0701 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.721033:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.721052:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 95 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.721084:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.721128:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 95 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.721144:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 95 pid: 6919 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302778.721161:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371129856:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302778.721322:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302778.731753:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45180 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371129856:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302778.731799:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302778.735897:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.739809:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350373760 00000100:00100000:0.0:1789302778.739830:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350373760 00010000:00010000:1.0:1789302778.740225:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 96 pid: 6919 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302778.740564:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:1.0:1789302778.740577:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09750050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350373760:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302778.740598:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302778.740606:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 96 pid: 6919 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.740621:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 96 pid: 6919 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.740649:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e13c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06fa expref: 96 pid: 6919 timeout: 715 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.740670:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302778.740676:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.740687:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.740696:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.740734:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302778.740822:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09750050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:2347:x1876219350373760:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 247us (4938us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302778.740830:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 329 00010000:00010000:0.0:1789302778.742388:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742406:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302778.742432:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 0 00010000:00010000:0.0:1789302778.742786:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742796:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742818:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742824:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302778.742829:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742838:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742867:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302778.742870:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302778.742876:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742883:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e151 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742968:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e14a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0701 expref: 95 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742979:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302778.742982:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302778.742986:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e14a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0701 expref: 95 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.742999:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea800, rc 0) 00000100:00100000:0.0:1789302778.743066:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:28724:x1876219350373632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 22265us (22800us total) trans 4294968006 rc 301/301 00000100:00100000:0.0:1789302778.743084:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1089 00000100:00100000:3.0:1789302778.749029:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.749701:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350374016 00000100:00100000:0.0:1789302778.749716:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350374016 00000100:00100000:0.0:1789302778.749789:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 217 00000100:00100000:0.0:1789302778.749802:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ec050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:28724:x1876219350374016:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302778.751639:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ec050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:28724:x1876219350374016:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1825us (2608us total) trans 4294968007 rc 0/0 00000100:00100000:0.0:1789302778.751656:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 217 00000100:00100000:3.0:1789302778.756005:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.756089:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350374144 00000100:00100000:0.0:1789302778.756102:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350374144 00000100:00100000:0.0:1789302778.756172:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1090 00000100:00100000:0.0:1789302778.756183:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:28724:x1876219350374144:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302778.756226:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302778.756259:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e15f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0708 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.756365:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e15f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0708 expref: 96 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.756385:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e15f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0708 expref: 96 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.756409:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501600) 00010000:00010000:0.0:1789302778.756413:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302778.756418:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:0.0:1789302778.756548:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:28724:x1876219350374144:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 366us (556us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302778.756555:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1090 00000100:00100000:3.0:1789302778.761185:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302778.767482:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350374400 00000100:00100000:2.0:1789302778.767528:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350374400 00000100:00100000:2.0:1789302778.767645:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1091 00000100:00100000:2.0:1789302778.767659:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:28724:x1876219350374400:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1789302778.767754:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.767777:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.767792:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.769812:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.769834:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.769842:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.769929:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302778.769935:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.769945:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.769965:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501600) 00010000:00010000:2.0:1789302778.769968:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302778.769974:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.769981:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e166 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302778.770095:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:28724:x1876219350374400:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 2434us (8920us total) trans 4294968008 rc 0/0 00000100:00100000:2.0:1789302778.770105:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1091 00000100:00100000:3.0:1789302778.773641:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.773902:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350374656 00000100:00100000:0.0:1789302778.773916:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350374656 00000100:00100000:0.0:1789302778.774003:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1092 00000100:00100000:0.0:1789302778.774017:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:28724:x1876219350374656:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302778.774660:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.774684:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.774700:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302778.774788:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968009 00000004:00010000:0.0:1789302778.774832:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e1a2081b reply state 00000000611f57e8 transno 4294968010 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302778.774843:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302778.775123:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:28724:x1876219350374656:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1106us (1504us total) trans 4294968010 rc 0/0 00000100:00100000:0.0:1789302778.775132:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1092 00000020:00080000:0.0:1789302778.775138:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1789302778.779841:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.779891:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.779902:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.779910:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302778.779917:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.779939:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.779961:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501600) 00010000:00010000:1.0:1789302778.779965:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302778.779972:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302778.779978:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e16d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302778.785002:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302778.785181:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350374912 00000100:00100000:2.0:1789302778.785196:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350374912 00000100:00100000:2.0:1789302778.785286:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 218 00000100:00100000:2.0:1789302778.785297:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:28724:x1876219350374912:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302778.785636:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:28724:x1876219350374912:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 339us (3292us total) trans 4294968011 rc 0/0 00000100:00100000:2.0:1789302778.785644:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 218 00000040:00080000:2.0:1789302778.829654:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302778.829667:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302778.830227:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1789302778.830238:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302778.919279:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302778.919722:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350375040 00000100:00100000:2.0:1789302778.919738:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350375040 00000100:00100000:2.0:1789302778.919814:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1093 00000100:00100000:2.0:1789302778.919828:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:28730:x1876219350375040:12345-192.168.203.11@tcp:101:cmp.0 00010000:00010000:2.0:1789302778.919874:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302778.919904:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e174 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a070f expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920007:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920027:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920043:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920061:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920068:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920075:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920212:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184e189 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920221:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184e189 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.920249:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184e189 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302778.921784:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 16384 00000004:00200000:3.0:1789302778.921813:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302778.921869:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184e189 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921892:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921900:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921909:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921917:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302778.921931:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921941:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921960:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:3.0:1789302778.921964:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302778.921970:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921978:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e17b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.921999:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922007:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922024:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922031:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302778.922034:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922041:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922050:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff960f03927400) 00010000:00010000:3.0:1789302778.922053:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302778.922056:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922063:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e182 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922116:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e174 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a070f expref: 102 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922133:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3e00/0x64fe43095184e189 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a070f expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.922141:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302778.922144:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:3.0:1789302778.922147:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3e00, rc 0) 00000100:00100000:3.0:1789302778.923167:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28730:x1876219350375040:12345-192.168.203.11@tcp:101:cmp.0 Request processed in 3330us (3910us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302778.923194:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1093 00000100:00100000:3.0:1789302778.927087:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302778.929484:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350375296 00000100:00100000:2.0:1789302778.929709:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350375296 00000100:00100000:3.0:1789302778.930139:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1094 00000100:00100000:3.0:1789302778.930156:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28730:x1876219350375296:12345-192.168.203.11@tcp:101:cmp.0 00010000:00010000:3.0:1789302778.930189:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302778.930234:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184e190 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0716 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.930570:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.930593:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.930608:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.930768:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.930777:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.930786:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.931149:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184e1a5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.931160:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184e1a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.931186:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184e1a5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302778.931209:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00010000:00010000:3.0:1789302778.931600:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.931610:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932063:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932071:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302778.932077:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932086:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932103:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302778.932106:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302778.932112:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932118:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e197 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932134:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932156:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932162:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932167:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302778.932170:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932176:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932184:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x81b47e8f (ffff960f0cdf2900) 00010000:00010000:3.0:1789302778.932186:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302778.932189:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932194:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e19e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932458:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184e190 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0716 expref: 104 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932478:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184e1a5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0716 expref: 103 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302778.932487:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501600) 00010000:00010000:3.0:1789302778.932490:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302778.932499:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302778.932632:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:28730:x1876219350375296:12345-192.168.203.11@tcp:101:cmp.0 Request processed in 2479us (5707us total) trans 4294968012 rc 0/0 00000100:00100000:3.0:1789302778.932643:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1094 00000100:00100000:0.0:1789302778.937810:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302778.938065:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350375552 00000100:00100000:2.0:1789302778.938079:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350375552 00000100:00100000:2.0:1789302778.938168:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 48 00000100:00100000:2.0:1789302778.938180:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:28730:x1876219350375552:12345-192.168.203.11@tcp:103:cmp.0 00010000:00010000:2.0:1789302778.938212:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1789302778.938246:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184e1a5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0716 expref: 103 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.938286:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e15f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0708 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.938299:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501600) 00010000:00010000:2.0:1789302778.938314:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302778.938704:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28730:x1876219350375552:12345-192.168.203.11@tcp:103:cmp.0 Request processed in 524us (908us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302778.938714:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 48 00000100:00100000:3.0:1789302778.941363:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.941435:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350375680 00000100:00100000:0.0:1789302778.941448:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350375680 00000100:00100000:0.0:1789302778.941550:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1095 00000100:00100000:0.0:1789302778.941562:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28730:x1876219350375680:12345-192.168.203.11@tcp:58:cmp.0 00010000:00010000:0.0:1789302778.941721:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.941741:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302778.941755:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789302778.942037:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].30, 136 off12272 00000100:00100000:0.0:1789302778.942890:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28730:x1876219350375680:12345-192.168.203.11@tcp:58:cmp.0 Request processed in 1323us (1545us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302778.942903:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1095 00000100:00100000:3.0:1789302778.946096:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302778.946490:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350375808 00000100:00100000:0.0:1789302778.948145:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350375808 00000100:00100000:2.0:1789302778.948596:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1096 00000100:00100000:2.0:1789302778.948617:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:28730:x1876219350375808:12345-192.168.203.11@tcp:101:cmp.0 00010000:00010000:2.0:1789302778.948648:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302778.948676:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a071d expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.948772:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.948782:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 4/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.948806:0:5843:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.948821:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 3/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.948839:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302778.948843:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302778.948847:0:5843:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 3/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302778.948867:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302778.949145:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:28730:x1876219350375808:12345-192.168.203.11@tcp:101:cmp.0 Request processed in 537us (3063us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302778.949154:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1096 00010000:00010000:0.0:1789302779.538063:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302779.852816:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302779.852829:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302779.852911:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 31 count 4 00000040:00080000:2.0:1789302779.852920:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302779.853692:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371129984:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302779.853938:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302779.878964:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371129984:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302779.880247:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302779.880398:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350375936 00000100:00100000:1.0:1789302779.880409:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350375936 00000100:00100000:1.0:1789302779.880480:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1097 00000100:00100000:1.0:1789302779.880492:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28731:x1876219350375936:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302779.880571:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x44:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302779.880722:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28731:x1876219350375936:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 231us (482us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302779.880751:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1097 00000100:00100000:2.0:1789302779.898022:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302779.898238:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350376064 00000100:00100000:0.0:1789302779.898251:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350376064 00000100:00100000:0.0:1789302779.898315:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1098 00000100:00100000:0.0:1789302779.898326:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28731:x1876219350376064:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1789302779.898426:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898612:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898626:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898644:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898668:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898676:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898701:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898710:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898716:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302779.898754:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00000004:00200000:0.0:1789302779.898758:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302779.898771:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898781:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898788:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898794:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898800:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302779.898803:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898817:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898831:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302779.898835:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302779.898841:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898849:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1c1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898864:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898871:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898877:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898882:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302779.898887:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898894:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898901:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x8b3c9e9 (ffff960f03501100) 00010000:00010000:0.0:1789302779.898905:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302779.898908:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898925:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1c8 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8b3c9e9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898940:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898946:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898951:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898956:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302779.898959:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898965:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898973:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302779.898976:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302779.898979:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.898984:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1cf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302779.899065:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28731:x1876219350376064:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 740us (1052us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302779.899074:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1098 00000100:00100000:3.0:1789302779.913029:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302779.913398:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350376320 00000100:00100000:0.0:1789302779.913410:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350376320 00000100:00100000:0.0:1789302779.913470:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1099 00000100:00100000:0.0:1789302779.913481:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:28731:x1876219350376320:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302779.913546:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302779.913567:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1d6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0724 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913681:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913696:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913707:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302779.913717:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00010000:00010000:0.0:1789302779.913816:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913824:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913831:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913837:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302779.913841:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913861:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913873:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302779.913876:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302779.913882:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.913888:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.914055:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1d6 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0724 expref: 44 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.914066:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302779.914069:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302779.914072:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1d6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0724 expref: 44 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.914083:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea800, rc 0) 00000100:00100000:0.0:1789302779.914142:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:28731:x1876219350376320:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 664us (1125us total) trans 4294968013 rc 301/301 00000100:00100000:0.0:1789302779.914150:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1099 00000100:00100000:3.0:1789302779.929030:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302779.929450:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350376448 00000100:00100000:2.0:1789302779.929460:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350376448 00000100:00100000:2.0:1789302779.929563:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1100 00000100:00100000:2.0:1789302779.929573:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:28731:x1876219350376448:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302779.929607:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x44:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302779.929718:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:28731:x1876219350376448:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 146us (696us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302779.929725:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1100 00000100:00100000:2.0:1789302779.941841:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302779.942069:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350376576 00000100:00100000:0.0:1789302779.942080:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350376576 00000100:00100000:0.0:1789302779.942151:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1101 00000100:00100000:0.0:1789302779.942163:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:28731:x1876219350376576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302779.942209:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302779.942229:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1eb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a072b expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942306:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942322:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942333:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942348:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942356:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942364:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942471:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663400/0x64fe43095184e200 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942480:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663400/0x64fe43095184e200 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942488:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663400/0x64fe43095184e200 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302779.942814:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 16384 00000004:00200000:0.0:1789302779.942825:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302779.942857:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663400/0x64fe43095184e200 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942871:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942877:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942884:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942890:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302779.942894:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942902:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942913:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:0.0:1789302779.942916:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302779.942921:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942927:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e1f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942941:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942948:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942965:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942971:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302779.942974:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942980:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942988:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xee467b96 (ffff960f03501100) 00010000:00010000:0.0:1789302779.942990:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302779.942993:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.942998:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e1f9 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xee467b96 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.943030:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e1eb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a072b expref: 45 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.943045:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663400/0x64fe43095184e200 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a072b expref: 44 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.943052:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03927a00) 00010000:00010000:0.0:1789302779.943055:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302779.943057:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01663400, rc 0) 00000100:00100000:0.0:1789302779.943117:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28731:x1876219350376576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 956us (1285us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302779.943146:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1101 00000100:00100000:3.0:1789302779.951265:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302779.951432:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350376832 00000100:00100000:0.0:1789302779.951443:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350376832 00000100:00100000:0.0:1789302779.951556:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1102 00000100:00100000:0.0:1789302779.951567:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:28731:x1876219350376832:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302779.951588:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302779.951609:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e207 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0732 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302779.959087:0:18427:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x17:0x0], index 0: 0 00010000:00010000:0.0:1789302779.960439:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e20e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.960461:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e20e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.960474:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e20e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.960834:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e207 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0732 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.961017:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e20e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0732 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.961027:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f03501100) 00010000:00010000:0.0:1789302779.961030:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302779.961047:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01663800, rc 0) 00000100:00100000:0.0:1789302779.962392:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28731:x1876219350376832:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 9560us (9871us total) trans 4294968014 rc 0/0 00000100:00100000:0.0:1789302779.962404:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1102 00000100:00100000:3.0:1789302779.970637:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302779.970743:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350377088 00000100:00100000:1.0:1789302779.970759:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350377088 00000100:00100000:1.0:1789302779.970832:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1103 00000100:00100000:1.0:1789302779.970842:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28731:x1876219350377088:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302779.970904:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302779.970931:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e20e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0732 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.970953:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f03501100) 00010000:00010000:1.0:1789302779.970964:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302779.970988:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971002:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971011:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971027:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971034:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971041:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971140:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971147:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971154:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971160:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302779.971164:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971171:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971180:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03927a00) 00010000:00010000:1.0:1789302779.971182:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302779.971186:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971202:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e21c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971212:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971218:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971224:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971235:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302779.971238:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971246:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971254:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x29bb6c05 (ffff960f10d24000) 00010000:00010000:1.0:1789302779.971256:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302779.971259:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.971265:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e223 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x42:0x0].0x29bb6c05 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302779.971356:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28731:x1876219350377088:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 515us (733us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302779.971364:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1103 00000100:00100000:2.0:1789302779.975941:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302779.976020:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350377216 00000100:00100000:1.0:1789302779.976030:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350377216 00000100:00100000:1.0:1789302779.976083:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1104 00000100:00100000:1.0:1789302779.976093:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28731:x1876219350377216:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302779.976113:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302779.976143:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e22a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0739 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.976178:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.976218:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.976229:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.976315:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e22a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.976333:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 46 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302779.976342:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302779.976345:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302779.976348:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6a00, rc 0) 00000100:00100000:1.0:1789302779.976403:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28731:x1876219350377216:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 312us (470us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302779.976411:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1104 00000100:00100000:3.0:1789302779.985432:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302779.985575:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350377472 00000100:00100000:0.0:1789302779.985590:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350377472 00000100:00100000:0.0:1789302779.985651:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1105 00000100:00100000:0.0:1789302779.985662:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28731:x1876219350377472:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302779.985743:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.985761:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.985772:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789302779.986043:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].14, 88 off9136 00000004:00080000:0.0:1789302779.986054:0:18427:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 616996551066ns, next at 617996549105ns, rc = 0 00000004:00010000:0.0:1789302779.986154:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001ae2947a reply state 00000000553714c8 transno 4294968015 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302779.986164:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302779.986270:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28731:x1876219350377472:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 608us (854us total) trans 4294968015 rc 0/0 00000100:00100000:0.0:1789302779.986278:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1105 00010000:00010000:2.0:1789302779.990565:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302779.990584:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302779.990592:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302779.990599:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302779.990605:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302779.990612:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302779.990632:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:2.0:1789302779.990652:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789302779.990658:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302779.990664:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e238 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302779.993953:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302779.994634:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350377600 00000100:00100000:0.0:1789302779.994650:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350377600 00000100:00100000:0.0:1789302779.994918:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1106 00000100:00100000:0.0:1789302779.994931:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28731:x1876219350377600:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302779.994984:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302779.995011:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0740 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.995091:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.995110:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302779.995133:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:0.0:1789302779.995137:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302779.995325:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01662800, rc 0) 00000100:00100000:0.0:1789302779.995410:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28731:x1876219350377600:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 481us (1472us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302779.995419:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1106 00000100:00100000:3.0:1789302779.999332:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.003350:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350377728 00000100:00100000:0.0:1789302780.003365:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350377728 00000100:00100000:0.0:1789302780.003468:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00100000:0.0:1789302780.003481:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350377728:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1789302780.003871:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350377728:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 393us (4550us total) trans 4294967354 rc 0/0 00000100:00100000:0.0:1789302780.003879:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00080000:3.0:1789302780.004580:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302780.006753:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.006829:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350377984 00000100:00100000:0.0:1789302780.006839:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350377984 00000100:00100000:0.0:1789302780.006908:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 219 00000100:00100000:0.0:1789302780.006919:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:28731:x1876219350377984:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302780.007256:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28731:x1876219350377984:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 338us (510us total) trans 4294968016 rc 0/0 00000100:00100000:0.0:1789302780.007265:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 219 00000100:00080000:3.0:1789302780.007425:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302780.015456:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.015704:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350378240 00000100:00100000:0.0:1789302780.015716:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350378240 00000100:00100000:0.0:1789302780.015780:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1107 00000100:00100000:0.0:1789302780.015791:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28731:x1876219350378240:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302780.015901:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28731:x1876219350378240:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 111us (455us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302780.015908:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1107 00000100:00100000:1.0:1789302780.019205:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.019773:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350378368 00000100:00100000:0.0:1789302780.019785:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350378368 00000100:00100000:0.0:1789302780.019844:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1108 00000100:00100000:0.0:1789302780.019855:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28731:x1876219350378368:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302780.019876:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302780.019897:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e246 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0747 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.019983:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e24d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.019997:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e24d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.020009:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e24d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302780.020056:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302780.020075:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e24d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.020099:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e246 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0747 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.020118:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e24d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0747 expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.020125:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:0.0:1789302780.020129:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1789302780.020133:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01663800, rc 0) 00000100:00100000:0.0:1789302780.020254:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28731:x1876219350378368:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 401us (1067us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302780.020262:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1108 00000100:00100000:2.0:1789302780.028794:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.028912:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350378496 00000100:00100000:1.0:1789302780.028922:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350378496 00000100:00100000:1.0:1789302780.028976:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:1.0:1789302780.028987:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350378496:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302780.029004:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302780.029027:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a074e expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302780.029072:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x17:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6aa697fb, blocks: 0x1 00010000:00010000:1.0:1789302780.029092:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=617s v=(0 0) 00010000:00010000:1.0:1789302780.029097:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302780.029107:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.029126:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.029139:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7800, rc 0) 00000100:00100000:1.0:1789302780.029209:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350378496:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 224us (424us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.029217:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:2.0:1789302780.063655:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.063864:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350378752 00000100:00100000:0.0:1789302780.063874:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350378752 00000100:00100000:0.0:1789302780.063936:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1109 00000100:00100000:0.0:1789302780.063946:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28731:x1876219350378752:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302780.064051:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28731:x1876219350378752:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 106us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302780.064059:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1109 00000100:00100000:3.0:1789302780.089665:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302780.089806:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350378880 00000100:00100000:2.0:1789302780.089816:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350378880 00000100:00100000:2.0:1789302780.089874:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 294 00000100:00100000:2.0:1789302780.089884:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:28731:x1876219350378880:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302780.089901:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302780.089919:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0755 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.089950:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.089960:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302780.089964:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302780.089970:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 4/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.090004:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.090036:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 12 pid: 13439 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302780.090052:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371130112:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302780.090158:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302780.100565:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371130112:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302780.100616:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 3/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.100641:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2a00, rc 0) 00000100:00100000:2.0:1789302780.100703:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:28731:x1876219350378880:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 10820us (11045us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302780.100712:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 294 00000100:00100000:3.0:1789302780.109568:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.109728:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350379008 00000100:00100000:1.0:1789302780.109738:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350379008 00010000:00010000:1.0:1789302780.109780:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 12 pid: 13439 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.109819:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:1.0:1789302780.109830:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350379008:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302780.109846:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302780.109879:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 12 pid: 13439 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.109914:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 12 pid: 13439 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.109937:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005acb256c/0x64fe43095184e254 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a074e expref: 12 pid: 13439 timeout: 717 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.109955:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302780.109960:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302780.109968:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.109978:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.110014:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.110027:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371130240:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302780.110251:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a300 going to sleep for 11 seconds 00000100:00100000:1.0:1789302780.120899:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371130240:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302780.120927:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302780.120995:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350379008:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 11166us (11435us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.121006:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:3.0:1789302780.121388:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.125825:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350379136 00000100:00100000:0.0:1789302780.125842:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350379136 00002000:00100000:0.0:1789302780.125913:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x17:0x0] (0->157) req@ffff960f3f70b800 x1876219350379136/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:356/0 lens 648/0 e 0 to 0 dl 1789302791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302780.125940:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f70b800 with x1876219350379136 ext(0->157) 00002000:00010000:0.0:1789302780.125947:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f70b800 00000100:00100000:0.0:1789302780.125988:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 101 00000100:00100000:0.0:1789302780.125998:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350379136:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1789302780.126986:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350379136:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 986us (5594us total) trans 4294967355 rc 0/0 00000100:00100000:0.0:1789302780.126995:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 101 00002000:00100000:0.0:1789302780.127004:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x17:0x0] (0->157) req@ffff960f3f70b800 x1876219350379136/t4294967355(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:356/0 lens 648/448 e 0 to 0 dl 1789302791 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302780.127012:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f70b800 with x1876219350379136 ext(0->157) 00002000:00010000:0.0:1789302780.127017:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f70b800 00000100:00100000:2.0:1789302780.137228:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302780.137345:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350379264 00000100:00100000:3.0:1789302780.137355:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350379264 00000100:00100000:3.0:1789302780.137420:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1110 00000100:00100000:3.0:1789302780.137430:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28731:x1876219350379264:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000100:00100000:3.0:1789302780.199633:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:28731:x1876219350379264:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 62191us (62400us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.199646:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1110 00000001:00080000:0.0:1789302780.203887:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968016 is committed 00000004:00080000:0.0:1789302780.203907:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 15 committed 00000001:00080000:0.0:1789302780.203929:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968010 is committed 00000020:00080000:0.0:1789302780.203946:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 20 batchid updated 00000001:00080000:0.0:1789302780.203959:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968011 is committed 00000001:00080000:0.0:1789302780.203972:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968012 is committed 00000100:00100000:1.0:1789302780.204205:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000020:00080000:1.0:1789302780.204237:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 20 00000020:00080000:1.0:1789302780.204247:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 20 00000100:00100000:0.0:1789302780.205877:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350379392 00000100:00100000:0.0:1789302780.205890:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350379392 00000100:00100000:0.0:1789302780.205948:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 295 00000100:00100000:0.0:1789302780.205957:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350379392:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000100:00100000:3.0:1789302780.207301:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302780.207329:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371130368:0@lo:2:osp-syn-0-0.0 00000100:00100000:3.0:1789302780.207387:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302780.207446:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371130368 00000100:00100000:3.0:1789302780.207453:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371130368 00000100:00100000:3.0:1789302780.207522:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1789302780.207530:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371130368:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:3.0:1789302780.207704:0:13439:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1789302780.207708:0:13439:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb mtime from disk: 0 -> 1789302779 00002000:00010000:3.0:1789302780.207711:0:13439:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1789302780.207713:0:13439:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb ctime from disk: 1789302779 -> 1789302779 00002000:00010000:3.0:1789302780.207715:0:13439:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1789302780.207754:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371130368:12345-0@lo:2:osp-syn-0-0.0 Request processed in 224us (366us total) trans 4294967356 rc 0/0 00000100:00100000:3.0:1789302780.207762:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00080000:3.0:1789302780.207782:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:3.0:1789302780.207815:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f70aa00/2, rc 0, transno 4294967356 00000100:00100000:3.0:1789302780.207822:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70aa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371130368:0@lo:2:osp-syn-0-0.0 00000001:00080000:2.0:1789302780.263585:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967355 is committed 00000100:00100000:3.0:1789302780.263639:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350379392:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 57676us (59440us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.263651:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 295 00000100:00100000:3.0:1789302780.266842:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.267008:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350379648 00000100:00100000:1.0:1789302780.267017:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350379648 00000100:00100000:1.0:1789302780.267207:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1111 00000100:00100000:1.0:1789302780.267218:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28731:x1876219350379648:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302780.267278:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302780.267304:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e24d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0747 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.267485:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302780.267491:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302780.267576:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.267619:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.267632:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302780.267962:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006d2dddfb reply state 00000000df8c373b transno 4294968017 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.267981:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302780.268082:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:28731:x1876219350379648:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 865us (1242us total) trans 4294968017 rc 0/0 00000100:00100000:1.0:1789302780.268089:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1111 00010000:00010000:3.0:1789302780.270315:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.270332:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.270339:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.270362:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302780.270368:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.270376:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.270392:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:3.0:1789302780.270396:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302780.270400:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.270406:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e262 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302780.270556:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.270862:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350379776 00000100:00100000:1.0:1789302780.270872:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350379776 00000100:00100000:1.0:1789302780.270951:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 297 00000100:00100000:1.0:1789302780.270961:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350379776:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1789302780.271150:0:6695:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:1.0:1789302780.271155:0:6695:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb mtime from disk: 1789302779 -> 1536580263 00002000:00010000:1.0:1789302780.271157:0:6695:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb atime from disk: 0 -> 1789302187 00002000:00010000:1.0:1789302780.271160:0:6695:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb ctime from disk: 1789302779 -> 1789302778 00002000:00010000:1.0:1789302780.271162:0:6695:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x17:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1789302780.271230:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350379776:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 269us (677us total) trans 4294967357 rc 0/0 00000100:00100000:1.0:1789302780.271238:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 297 00000100:00100000:3.0:1789302780.273921:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.274405:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350380032 00000100:00100000:0.0:1789302780.274415:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350380032 00000100:00100000:0.0:1789302780.274480:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1112 00000100:00100000:0.0:1789302780.274537:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28731:x1876219350380032:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:0.0:1789302780.274614:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.274630:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 46 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.274639:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.274673:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 46 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.274687:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 46 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302780.274701:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371130496:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302780.274785:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebdc00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302780.279037:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371130496:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302780.279068:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302780.280749:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.281411:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350380160 00000100:00100000:1.0:1789302780.281422:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350380160 00010000:00010000:1.0:1789302780.281460:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 47 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.281535:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:1.0:1789302780.281546:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350380160:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302780.281566:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302780.281573:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 47 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.281585:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 47 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.282255:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e231 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x1f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0739 expref: 47 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.282399:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302780.282404:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.282413:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.282421:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.282519:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302780.282576:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350380160:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1031us (1831us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.282584:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 331 00010000:00010000:2.0:1789302780.282683:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282692:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282725:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282743:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282749:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282754:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302780.282759:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282766:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282775:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:2.0:1789302780.282777:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302780.282781:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.282786:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e269 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302780.282830:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:28731:x1876219350380032:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 8298us (8917us total) trans 0 rc -22/-22 00000100:00100000:2.0:1789302780.282837:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1112 00000100:00100000:3.0:1789302780.287023:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.287123:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350380288 00000100:00100000:1.0:1789302780.287134:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350380288 00000100:00100000:1.0:1789302780.287202:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 298 00000100:00100000:1.0:1789302780.287213:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350380288:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302780.287259:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.287290:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.287299:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302780.287303:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302780.287308:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.287359:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.287372:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 11 pid: 8571 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.287385:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371130624:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302780.287468:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335be900 going to sleep for 11 seconds 00000100:00100000:3.0:1789302780.291482:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.291649:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371130624:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302780.291676:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302780.291784:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350380416 00000100:00100000:2.0:1789302780.291791:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350380416 00010000:00010000:3.0:1789302780.292233:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 12 pid: 8571 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302780.292370:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 332 00000100:00100000:3.0:1789302780.292380:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350380416:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302780.292394:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302780.292416:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 12 pid: 8571 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.292451:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 12 pid: 8571 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.292473:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e25b lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0755 expref: 12 pid: 8571 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.292491:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302780.292495:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302780.292533:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.292542:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.292567:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302780.292625:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350380416:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 247us (1153us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.292632:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 332 00010000:00010000:1.0:1789302780.293022:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293030:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293050:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293055:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293071:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293075:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302780.293078:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293085:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293097:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.293114:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e270 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.293175:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350380288:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 5965us (6161us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.293181:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 298 00000100:00100000:3.0:1789302780.296215:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.296325:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350380672 00000100:00100000:0.0:1789302780.296335:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350380672 00000100:00100000:0.0:1789302780.296445:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1113 00000100:00100000:0.0:1789302780.296456:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:28731:x1876219350380672:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302780.296580:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.296612:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.296624:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.296844:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.296857:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.296865:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 4/0,1 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.296897:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.296911:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 46 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302780.296924:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371130752:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302780.297003:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebdc00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302780.305690:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.305793:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350380800 00000100:00100000:1.0:1789302780.305802:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350380800 00010000:00010000:1.0:1789302780.305842:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 47 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.305884:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 333 00000100:00100000:1.0:1789302780.305910:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350380800:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302780.305925:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302780.305933:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 47 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.305946:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 47 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.305966:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302780.305972:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,1 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.305981:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.305989:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.306009:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302780.306072:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350380800:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 164us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.306091:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 333 00010000:00010000:3.0:1789302780.306277:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e23f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0740 expref: 46 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302780.306296:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371130752:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302780.306317:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.306337:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.306343:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302780.306575:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968018 00000001:00080000:3.0:1789302780.306730:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968019 00000001:00080000:3.0:1789302780.306777:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968020 00000100:00100000:3.0:1789302780.306856:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371130880:0@lo:1000:mdt00_004.0 00000100:00100000:3.0:1789302780.306903:0:18427:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302780.306929:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302780.307398:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371130880 00000100:00100000:1.0:1789302780.307406:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371130880 00000100:00100000:1.0:1789302780.307475:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 45 00000100:00100000:1.0:1789302780.307485:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:18427:x1876219371130880:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789302780.307589:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:18427:x1876219371130880:12345-0@lo:1000:mdt00_004.0 Request processed in 106us (685us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.307612:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 45 00000100:00100000:1.0:1789302780.307738:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7480 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371130880:0@lo:1000:mdt00_004.0 00000001:00080000:1.0:1789302780.308168:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968021 00000100:00100000:1.0:1789302780.308242:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371131008:0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789302780.308435:0:18427:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302780.308460:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335bef00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302780.308477:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371131008 00000100:00100000:1.0:1789302780.308481:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371131008 00000100:00100000:1.0:1789302780.308530:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 46 00000100:00100000:1.0:1789302780.308537:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:18427:x1876219371131008:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789302780.308676:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:18427:x1876219371131008:12345-0@lo:1000:mdt00_004.0 Request processed in 139us (241us total) trans 4294967358 rc 0/0 00000100:00100000:1.0:1789302780.308683:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 46 00000100:00100000:1.0:1789302780.308715:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d880 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371131008:0@lo:1000:mdt00_004.0 00010000:00010000:1.0:1789302780.308744:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308753:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308760:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308766:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302780.308771:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308779:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308800:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302780.308802:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302780.308807:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 2/0,0 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308814:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e27e lrc: 0/0,0 mode: --/EX res: [0x200000402:0x1f:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308843:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308849:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308855:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308860:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eb800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302780.308863:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308868:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308874:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:1.0:1789302780.308876:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302780.308882:0:18427:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 92 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308892:0:18427:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 92 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308907:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 92 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.308930:0:18427:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 92 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.308941:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371131136:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302780.308987:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cfc00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302780.314940:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371131136:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302780.314972:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.314982:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e1b3 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315016:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315030:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315041:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315051:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315057:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315094:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315101:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302780.315105:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315113:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315125:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:1.0:1789302780.315128:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302780.315132:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315139:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e285 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315149:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315155:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315160:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315164:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302780.315168:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315185:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315201:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:1.0:1789302780.315204:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302780.315207:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.315212:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e277 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.315445:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28731:x1876219350380672:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 18990us (19237us total) trans 4294968022 rc 0/0 00000100:00100000:1.0:1789302780.315456:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1113 00000020:00080000:1.0:1789302780.315462:0:18427:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000040:00080000:2.0:1789302780.316381:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302780.316388:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302780.317212:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:2.0:1789302780.317226:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302780.321148:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.321239:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350381056 00000100:00100000:1.0:1789302780.321249:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350381056 00000100:00100000:1.0:1789302780.321325:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 220 00000100:00100000:1.0:1789302780.321336:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:28731:x1876219350381056:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1789302780.322036:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x1f:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1789302780.322141:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968023 00010000:00010000:1.0:1789302780.322188:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.322213:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.322224:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.322242:0:5846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.322247:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.322252:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.322265:0:5846:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1789302780.322357:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:28731:x1876219350381056:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1022us (1215us total) trans 4294968024 rc 0/0 00000100:00100000:1.0:1789302780.322365:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 220 00000020:00080000:1.0:1789302780.322370:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1789302780.323120:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302780.323128:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.323136:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302780.323140:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.323149:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.323163:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x1f:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:2.0:1789302780.323166:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789302780.323171:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 1/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.323177:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e28c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x1f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.323207:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302780.323209:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302780.323211:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302780.327860:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302780.327942:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350381184 00000100:00100000:2.0:1789302780.327952:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350381184 00000100:00100000:2.0:1789302780.328026:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 221 00000100:00100000:2.0:1789302780.328037:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28730:x1876219350381184:12345-192.168.203.11@tcp:49:cmp.0 00000100:00100000:2.0:1789302780.328170:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28730:x1876219350381184:12345-192.168.203.11@tcp:49:cmp.0 Request processed in 135us (318us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302780.328177:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 221 00000100:00080000:3.0:1789302780.328647:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302780.332735:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302780.332882:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350381312 00000100:00100000:2.0:1789302780.332892:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350381312 00000100:00100000:2.0:1789302780.332969:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 299 00000100:00100000:2.0:1789302780.332979:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:28730:x1876219350381312:12345-192.168.203.11@tcp:101:cmp.0 00010000:00010000:2.0:1789302780.332997:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302780.333021:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a075c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302780.333045:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x17:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6aa695ab, ctime: 0x6aa697fa, blocks: 0xb 00010000:00010000:2.0:1789302780.333068:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=617s v=(0 0) 00010000:00010000:2.0:1789302780.333073:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302780.333083:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.333098:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.333110:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3600, rc 0) 00000100:00100000:2.0:1789302780.333173:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:28730:x1876219350381312:12345-192.168.203.11@tcp:101:cmp.0 Request processed in 196us (448us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302780.333180:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 299 00000100:00080000:3.0:1789302780.334056:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302780.341994:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302780.342097:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350381440 00000100:00100000:2.0:1789302780.342108:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350381440 00002000:00100000:2.0:1789302780.342153:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x17:0x0] (0->4095) req@ffff960f3a9e1500 x1876219350381440/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:356/0 lens 488/0 e 0 to 0 dl 1789302791 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302780.342162:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3a9e1500 with x1876219350381440 ext(0->4095) 00002000:00010000:2.0:1789302780.342184:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3a9e1500 00000100:00100000:2.0:1789302780.342232:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 102 00000100:00100000:2.0:1789302780.342241:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350381440:12345-192.168.203.11@tcp:3:cmp.0 00000100:00100000:2.0:1789302780.343565:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350381440:12345-192.168.203.11@tcp:3:cmp.0 Request processed in 1319us (1572us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1789302780.343576:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 102 00002000:00100000:2.0:1789302780.343586:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x17:0x0] (0->4095) req@ffff960f3a9e1500 x1876219350381440/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:356/0 lens 488/4536 e 0 to 0 dl 1789302791 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cmp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302780.343596:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3a9e1500 with x1876219350381440 ext(0->4095) 00002000:00010000:2.0:1789302780.343601:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3a9e1500 00000100:00100000:1.0:1789302780.352863:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.353032:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350381696 00000100:00100000:0.0:1789302780.353042:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350381696 00000100:00100000:0.0:1789302780.353119:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 222 00000100:00100000:0.0:1789302780.353141:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:28730:x1876219350381696:12345-192.168.203.11@tcp:35:cmp.0 00000001:00080000:0.0:1789302780.353420:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968025 00000100:00100000:0.0:1789302780.353553:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:28730:x1876219350381696:12345-192.168.203.11@tcp:35:cmp.0 Request processed in 425us (698us total) trans 4294968026 rc 0/0 00000100:00100000:0.0:1789302780.353562:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 222 00000020:00080000:0.0:1789302780.353567:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789302780.389973:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.390933:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350381824 00000100:00100000:1.0:1789302780.390944:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350381824 00000100:00100000:1.0:1789302780.391011:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1114 00000100:00100000:1.0:1789302780.391021:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:28732:x1876219350381824:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302780.391047:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302780.391068:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e29a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0763 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.391134:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.391148:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.391178:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.391203:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.391210:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.391217:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392081:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 3/1,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392094:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392102:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302780.392138:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00000004:00200000:1.0:1789302780.392142:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302780.392161:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392173:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392182:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392210:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392216:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392223:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302780.392227:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392235:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.392247:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03927a00) 00010000:00010000:1.0:1789302780.392251:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302780.392256:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395626:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2a1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395656:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395663:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395670:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395676:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302780.395701:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395851:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395868:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x81b47e8f (ffff960f02dfd900) 00010000:00010000:1.0:1789302780.395872:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302780.395876:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395883:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e2a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x42:0x0].0x81b47e8f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395935:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e29a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x42:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 97 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395951:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 96 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.395957:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:1.0:1789302780.395960:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302780.395962:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6a00, rc 0) 00000100:00100000:1.0:1789302780.396105:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:28732:x1876219350381824:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 5083us (6140us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.396114:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1114 00000100:00100000:1.0:1789302780.408151:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302780.408569:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350381952 00000100:00100000:3.0:1789302780.408579:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350381952 00000100:00100000:3.0:1789302780.408633:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1115 00000100:00100000:3.0:1789302780.408642:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:28732:x1876219350381952:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302780.408663:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302780.408682:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e2b6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a076a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.408743:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184e2bd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x42:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.408757:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184e2bd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.408790:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184e2bd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302780.408820:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x42:0x0]: returning size 16384 00000004:00200000:3.0:1789302780.408822:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302780.408835:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184e2bd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.408863:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e2b6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a076a expref: 98 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.408876:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0a00/0x64fe43095184e2bd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x42:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a076a expref: 97 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302780.408882:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x42:0x0].0x0 (ffff960f03927a00) 00010000:00010000:3.0:1789302780.408884:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302780.408886:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0a00, rc 0) 00000100:00100000:3.0:1789302780.408933:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:28732:x1876219350381952:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 293us (791us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.408940:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1115 00000100:00100000:0.0:1789302780.416796:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302780.416907:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350382208 00000100:00100000:2.0:1789302780.416917:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350382208 00000100:00100000:2.0:1789302780.418622:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1116 00000100:00100000:2.0:1789302780.418641:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:28732:x1876219350382208:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302780.418663:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302780.418686:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e2c4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0771 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.418795:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e2cb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.418809:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e2cb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.418820:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e2cb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302780.418852:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x44:0x0]: returning size 158 00010000:00010000:2.0:1789302780.418971:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e2c4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0771 expref: 100 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.418991:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e2cb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0771 expref: 99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.418998:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:2.0:1789302780.419001:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1789302780.419004:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3800, rc 0) 00000100:00100000:2.0:1789302780.419075:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:28732:x1876219350382208:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 442us (2371us total) trans 4294968027 rc 0/0 00000100:00100000:2.0:1789302780.419083:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1116 00000100:00100000:3.0:1789302780.423968:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302780.424039:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350382336 00000100:00100000:2.0:1789302780.424049:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350382336 00000100:00100000:2.0:1789302780.424148:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:2.0:1789302780.424158:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350382336:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:2.0:1789302780.424213:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.424233:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.424242:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302780.424294:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302780.424299:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 4/0,1 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.424351:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 12 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.424365:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 12 pid: 6697 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302780.424379:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcddf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371131264:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302780.424694:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302780.440685:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcddf80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371131264:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302780.440715:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302780.441179:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.441405:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350382464 00000100:00100000:1.0:1789302780.441414:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350382464 00010000:00010000:1.0:1789302780.441451:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 13 pid: 6697 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.441539:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 334 00000100:00100000:1.0:1789302780.441549:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350382464:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302780.441563:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302780.441586:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 13 pid: 6697 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.441599:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 13 pid: 6697 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.441620:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e293 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a075c expref: 13 pid: 6697 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.441636:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302780.441641:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302780.441646:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.441655:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.441680:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302780.441735:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350382464:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 188us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302780.441742:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 334 00010000:00010000:0.0:1789302780.441825:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441835:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441869:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441878:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441884:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441891:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302780.441895:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441914:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441926:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.441934:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184e2d9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302780.441996:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350382336:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 17839us (18037us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302780.442003:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:3.0:1789302780.457622:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.457780:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350382720 00000100:00100000:1.0:1789302780.457789:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350382720 00000100:00100000:1.0:1789302780.457847:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 223 00000100:00100000:1.0:1789302780.457858:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076eec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:28732:x1876219350382720:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:1.0:1789302780.457942:0:5846:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000401:0x44:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e2cb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0771 expref: 99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789302780.458176:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968028 00000001:00080000:1.0:1789302780.458741:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968029 00010000:00010000:1.0:1789302780.458801:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.458818:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.458825:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 102 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.458832:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.458868:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 102 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.458883:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 102 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.458897:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371131392:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302780.458991:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302780.459008:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 102 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302780.459018:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371131520:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302780.459044:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cfc00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302780.465665:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302780.465761:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302780.467141:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350382848 00000100:00100000:0.0:1789302780.467154:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350382848 00010000:00010000:0.0:1789302780.467212:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 103 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302780.467254:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350382976 00000100:00100000:0.0:1789302780.467258:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350382976 00010000:00010000:0.0:1789302780.467280:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 104 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302780.467308:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:0.0:1789302780.467317:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09752c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350382848:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302780.467331:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302780.467354:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 104 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467368:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 104 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467465:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302780.467470:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467479:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467486:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302780.467571:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09752c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350382848:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 255us (1915us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302780.467579:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:0.0:1789302780.467641:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 336 00000100:00100000:0.0:1789302780.467648:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2347:x1876219350382976:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302780.467657:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302780.467662:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 5/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 103 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467672:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 103 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467685:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302780.467689:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467704:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467712:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.467730:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302780.467750:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2347:x1876219350382976:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 102us (1988us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302780.467756:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 336 00010000:00010000:2.0:1789302780.473202:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e2af lrc: 0/0,0 mode: --/PR res: [0x200000401:0x44:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0763 expref: 102 pid: 6919 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302780.473230:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371131392:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302780.473249:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e1ba lrc: 0/0,0 mode: --/CR res: [0x200000401:0x44:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a071d expref: 101 pid: 5843 timeout: 717 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302780.473259:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371131520:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302780.473671:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.473681:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302780.473707:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302780.473828:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968030 00000001:00080000:2.0:1789302780.474067:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968031 00000100:00100000:2.0:1789302780.474164:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcddf80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371131648:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302780.474235:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302780.474285:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43e40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302780.474495:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371131648 00000100:00100000:3.0:1789302780.474527:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371131648 00000100:00100000:3.0:1789302780.474572:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:3.0:1789302780.474582:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5846:x1876219371131648:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1789302780.474674:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5846:x1876219371131648:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 93us (438us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.474681:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00000100:00100000:0.0:1789302780.475653:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcddf80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371131648:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:0.0:1789302780.475942:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968032 00000100:00100000:0.0:1789302780.476017:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f708380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371131776:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1789302780.476064:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302780.476103:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25ebd240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302780.476150:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371131776 00000100:00100000:3.0:1789302780.476160:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371131776 00000100:00100000:3.0:1789302780.476211:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:3.0:1789302780.476220:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5846:x1876219371131776:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1789302780.476724:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+8:5846:x1876219371131776:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 504us (658us total) trans 4294967359 rc 0/0 00000100:00100000:3.0:1789302780.476732:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00000100:00100000:0.0:1789302780.477544:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f708380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371131776:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:0.0:1789302780.477618:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968033 00010000:00010000:0.0:1789302780.477662:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.477671:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.477677:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.477682:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302780.477686:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.477706:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.477721:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302780.477724:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1789302780.477729:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302780.477735:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e2e0 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x44:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302780.478972:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x45:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1789302780.479182:0:5846:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].15, 64 off9200 00000001:00080000:0.0:1789302780.479207:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968034 00010000:00010000:0.0:1789302780.479257:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x44:0x0].0x0 (ffff960f03501000) 00010000:00010000:0.0:1789302780.479581:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302780.479591:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e2cb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x44:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0771 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302780.479765:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076eec50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:28732:x1876219350382720:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 21909us (22152us total) trans 4294968035 rc 0/0 00000100:00100000:0.0:1789302780.479775:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 223 00000020:00080000:0.0:1789302780.479780:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00010000:00010000:3.0:1789302780.557230:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302780.941063:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371132160:0@lo:400:kworker.0 00000100:00100000:2.0:1789302780.941143:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302780.941217:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302780.941223:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371132032:0@lo:400:kworker.0 00000100:00100000:2.0:1789302780.941246:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302780.941279:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132160 00000100:00100000:2.0:1789302780.941287:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132160 00000100:00100000:2.0:1789302780.941332:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132032 00000100:00100000:2.0:1789302780.941335:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132032 00000100:00100000:2.0:1789302780.941354:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:2.0:1789302780.941361:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371132160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302780.941413:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371132160:12345-0@lo:400:kworker.0 Request processed in 51us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302780.941420:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:2.0:1789302780.941436:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789302780.941442:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371132032:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302780.941483:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371132032:12345-0@lo:400:kworker.0 Request processed in 41us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302780.941533:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789302780.941570:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371132032:0@lo:400:kworker.0 00000100:00100000:2.0:1789302780.941581:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcddf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371132160:0@lo:400:kworker.0 00000100:00100000:0.0:1789302780.941632:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371132288:0@lo:400:kworker.0 00000100:00100000:0.0:1789302780.941666:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302780.941670:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371131904:0@lo:400:kworker.0 00000100:00100000:3.0:1789302780.941717:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302780.941749:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371131904 00000100:00100000:0.0:1789302780.941754:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371131904 00000100:00100000:3.0:1789302780.941755:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371132416:0@lo:400:kworker.0 00000100:00100000:3.0:1789302780.941786:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302780.941793:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 566 00000100:00100000:3.0:1789302780.941820:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132416 00000100:00100000:0.0:1789302780.941820:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371131904:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302780.941828:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132416 00000100:00100000:3.0:1789302780.941864:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789302780.941871:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371131904:12345-0@lo:400:kworker.0 Request processed in 54us (152us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.941874:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371132416:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302780.941886:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 566 00000100:00100000:3.0:1789302780.941919:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371132416:12345-0@lo:400:kworker.0 Request processed in 46us (131us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.941927:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:0.0:1789302780.941931:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371131904:0@lo:400:kworker.0 00000100:00100000:0.0:1789302780.941945:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371132416:0@lo:400:kworker.0 00000100:00080000:3.0:1789302780.941949:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302780.941955:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302780.941965:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132288 00000100:00100000:3.0:1789302780.941969:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132288 00000100:00100000:3.0:1789302780.942003:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:3.0:1789302780.942010:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371132288:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302780.942056:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371132288:12345-0@lo:400:kworker.0 Request processed in 46us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302780.942061:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00080000:3.0:1789302780.942074:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302780.942880:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095adc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371132288:0@lo:400:kworker.0 00000100:00100000:1.0:1789302781.195777:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302781.195827:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302781.195860:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302781.195888:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302781.195904:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302781.195919:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302781.195946:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302781.195973:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350383360 00000100:00100000:1.0:1789302781.195983:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350383360 00000100:00100000:1.0:1789302781.196027:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350383872 00000100:00100000:1.0:1789302781.196030:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350383872 00000100:00100000:1.0:1789302781.196053:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350383488 00000100:00100000:1.0:1789302781.196056:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350383488 00000100:00100000:1.0:1789302781.196070:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350383744 00000100:00100000:1.0:1789302781.196073:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350383744 00000100:00100000:1.0:1789302781.196086:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 402 00000100:00100000:1.0:1789302781.196095:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350383360:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302781.196160:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350383360:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 65us (331us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302781.196195:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 402 00000100:00100000:1.0:1789302781.196214:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 403 00000100:00100000:1.0:1789302781.196221:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350383872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302781.196248:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350383872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (359us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302781.196273:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 403 00000100:00100000:1.0:1789302781.196286:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:1.0:1789302781.196292:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350383488:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302781.196315:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350383488:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (394us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302781.196545:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 405 00000100:00100000:2.0:1789302781.196556:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350383744:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302781.196584:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350383744:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (637us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302781.196590:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 405 00000100:00100000:1.0:1789302781.196799:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:0.0:1789302781.196871:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350383104 00000100:00100000:0.0:1789302781.196877:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350383104 00000100:00100000:0.0:1789302781.196918:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:0.0:1789302781.196926:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350383104:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302781.196957:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350383104:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (1188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302781.196963:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:0.0:1789302781.196992:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350383616 00000100:00100000:0.0:1789302781.196995:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350383616 00000100:00100000:0.0:1789302781.197027:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350383232 00000100:00100000:0.0:1789302781.197030:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350383232 00000100:00100000:0.0:1789302781.197052:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 408 00000100:00100000:0.0:1789302781.197058:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:2347:x1876219350383616:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302781.197089:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:2347:x1876219350383616:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (1228us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302781.197095:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 408 00000100:00100000:0.0:1789302781.197107:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 409 00000100:00100000:0.0:1789302781.197126:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350383232:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302781.197154:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350383232:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (1249us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302781.197160:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 409 00000100:00080000:3.0:1789302781.197858:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302781.453438:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371132544:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302781.453548:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302781.453625:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132544 00000100:00100000:2.0:1789302781.453633:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132544 00000100:00100000:2.0:1789302781.453680:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1789302781.453690:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371132544:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302781.453724:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 78184448 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302781.453770:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371132544:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302781.453777:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1789302781.453824:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371132544:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302781.580757:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1789302781.966241:0:6999:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 618976736340ns, next at 619976734297ns, rc = 0 00000100:00100000:2.0:1789302781.966381:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371132672:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302781.966446:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302781.967859:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132672 00000100:00100000:1.0:1789302781.967870:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132672 00000100:00100000:1.0:1789302781.967925:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 177 00000100:00100000:1.0:1789302781.967935:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3301:x1876219371132672:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302781.967956:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3858759680 avail 3855613952 00000100:00100000:1.0:1789302781.968022:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3301:x1876219371132672:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (1576us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302781.968028:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 177 00000100:00100000:3.0:1789302781.970112:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371132672:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302782.605920:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302783.632319:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302784.652851:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302785.603003:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968035 is committed 00000020:00080000:1.0:1789302785.603032:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f25ebd3c0 refcount 2 committed 0 result 0 batchid 22 00000020:00080000:1.0:1789302785.603039:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f085af900 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f25ebd6c0 00000020:00080000:1.0:1789302785.603043:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f085af800 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3b0b9400 00000001:00080000:1.0:1789302785.603075:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968018 is committed 00000004:00080000:1.0:1789302785.603084:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 16 committed 00000001:00080000:1.0:1789302785.603113:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968024 is committed 00000020:00080000:1.0:1789302785.603128:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f335becc0 refcount 2 committed 0 result 0 batchid 21 00000020:00080000:1.0:1789302785.603131:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f034b9400 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f335be900 00000020:00080000:1.0:1789302785.603134:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f034b9f00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f098af400 00000020:00080000:1.0:1789302785.603184:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 22 committed batchid 20 00000020:00080000:1.0:1789302785.603576:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 20 00000100:00100000:1.0:1789302785.603713:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371132800:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789302785.603802:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302785.607861:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132800 00000100:00100000:0.0:1789302785.607883:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132800 00000100:00100000:0.0:1789302785.607971:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:0.0:1789302785.607983:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+8:3302:x1876219371132800:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789302785.608024:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:23 count 1 00002000:00010000:0.0:1789302785.608073:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x17:0x0] initial LVB size: 158, mtime: 0x5b965aa7, atime: 0x6aa695ab, ctime: 0x6aa697fa, blocks: 0xb 00010000:00010000:0.0:1789302785.608082:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302785.608105:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=622s v=(0 0) 00010000:00010000:0.0:1789302785.608110:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302785.608118:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302785.608149:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302785.608173:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302785.608181:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302785.608231:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302785.608309:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302785.608319:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302785.608327:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302785.608332:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302785.608342:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302785.608362:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302785.608369:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000040326cc/0x64fe43095184e2e7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302785.608401:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302785.608403:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302785.608405:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302785.608897:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+9:3302:x1876219371132800:12345-0@lo:6:osp-syn-0-0.0 Request processed in 914us (5093us total) trans 4294967360 rc 0/0 00000100:00100000:0.0:1789302785.608906:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00080000:0.0:1789302785.608939:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:1.0:1789302785.608975:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e0114a680/2, rc 0, transno 4294967360 00000100:00100000:1.0:1789302785.608984:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371132800:0@lo:6:osp-syn-0-0.0 00000001:00080000:2.0:1789302785.631431:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967358 is committed 00000001:00080000:2.0:1789302785.631471:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967357 is committed 00000001:00080000:2.0:1789302785.631479:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967359 is committed 00010000:00010000:3.0:1789302785.679287:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302786.064732:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371132928:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.064828:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302786.064888:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371133440:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.064922:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302786.064955:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302786.064960:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371133312:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.064988:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302786.065021:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371133056:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.065046:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302786.065064:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371133568:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.065084:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302786.065102:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302786.065107:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371133184:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.065127:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302786.065170:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371132928 00000100:00100000:2.0:1789302786.065179:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371132928 00000100:00100000:2.0:1789302786.065235:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 568 00000100:00100000:2.0:1789302786.065245:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371132928:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.065320:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371132928:12345-0@lo:400:kworker.0 Request processed in 76us (491us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302786.065328:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 568 00000100:00100000:2.0:1789302786.065365:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371133312 00000100:00100000:2.0:1789302786.065369:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371133312 00000100:00100000:0.0:1789302786.065799:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371133440 00000100:00100000:0.0:1789302786.065804:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371133440 00000100:00100000:0.0:1789302786.065848:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371133056 00000100:00100000:0.0:1789302786.065852:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371133056 00000100:00100000:0.0:1789302786.065871:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371133184 00000100:00100000:0.0:1789302786.065874:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371133184 00000100:00100000:0.0:1789302786.065891:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:0.0:1789302786.065899:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371133440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302786.065944:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371133440:12345-0@lo:400:kworker.0 Request processed in 46us (1021us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302786.065950:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:0.0:1789302786.065964:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1789302786.065969:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371133056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302786.065999:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371133056:12345-0@lo:400:kworker.0 Request processed in 30us (952us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302786.066006:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1789302786.066015:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1789302786.066020:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371133184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302786.066045:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371133184:12345-0@lo:400:kworker.0 Request processed in 25us (917us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302786.066060:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00080000:0.0:1789302786.066078:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302786.066162:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371133568 00000100:00100000:2.0:1789302786.066173:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371133568 00000100:00100000:2.0:1789302786.066216:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789302786.066222:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371133312:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.066256:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371133312:12345-0@lo:400:kworker.0 Request processed in 34us (1267us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302786.066262:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1789302786.066275:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789302786.066281:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371133568:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.066319:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371133568:12345-0@lo:400:kworker.0 Request processed in 39us (1234us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302786.066326:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:2.0:1789302786.067206:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371133184:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.067232:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095aed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371133056:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.067244:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095aca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371133568:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.068003:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967359 gen 2 00000004:00080000:2.0:1789302786.068013:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f70aa00, transno 4294967356 00000004:00080000:2.0:1789302786.068029:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967358 committed transno 4294967359 00000004:00080000:2.0:1789302786.068050:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967359 committed transno 4294967359 00000100:00100000:2.0:1789302786.068071:0:3303:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff960e0114a680 x1876219371132800/t4294967360(4294967360) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789302801 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000100:00100000:2.0:1789302786.068083:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ae680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371133312:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.068096:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371132928:0@lo:400:kworker.0 00000100:00100000:2.0:1789302786.068109:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371133440:0@lo:400:kworker.0 00000100:00080000:0.0:1789302786.071558:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302786.317855:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302786.317909:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302786.317946:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302786.317972:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384128 00000100:00100000:0.0:1789302786.317987:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384128 00000100:00100000:3.0:1789302786.317998:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302786.318022:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302786.318045:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302786.318071:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302786.318077:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384512 00000100:00100000:0.0:1789302786.318082:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384512 00000100:00100000:1.0:1789302786.318104:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384384 00000100:00100000:1.0:1789302786.318111:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384384 00000100:00100000:0.0:1789302786.318113:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 413 00000100:00100000:0.0:1789302786.318126:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:2346:x1876219350384128:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302786.318128:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 414 00000100:00100000:3.0:1789302786.318144:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2346:x1876219350384512:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302786.318168:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384640 00000100:00100000:1.0:1789302786.318172:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384640 00000100:00100000:1.0:1789302786.318194:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384256 00000100:00100000:1.0:1789302786.318197:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384256 00000100:00100000:1.0:1789302786.318211:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384768 00000100:00100000:1.0:1789302786.318213:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384768 00000100:00100000:1.0:1789302786.318227:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 408 00000100:00100000:1.0:1789302786.318235:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350384384:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302786.318304:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:2346:x1876219350384128:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 178us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302786.318306:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2346:x1876219350384512:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 162us (255us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302786.318319:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 413 00000100:00100000:0.0:1789302786.318360:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384000 00000100:00100000:0.0:1789302786.318364:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384000 00000100:00100000:2.0:1789302786.318369:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 409 00000100:00100000:0.0:1789302786.318394:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 569 00000100:00100000:0.0:1789302786.318404:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350384000:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302786.318411:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350384640:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302786.318434:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350384000:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302786.318441:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 569 00000100:00100000:2.0:1789302786.318454:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350384640:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 47us (507us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302786.318462:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 409 00000100:00100000:2.0:1789302786.318484:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 410 00000100:00100000:2.0:1789302786.318493:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350384256:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302786.318536:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 411 00000100:00100000:2.0:1789302786.318552:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350384256:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 61us (528us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302786.318559:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 410 00000100:00080000:2.0:1789302786.318596:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302786.318602:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302786.318604:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302786.318650:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350384768:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302786.318784:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350384768:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 240us (711us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302786.318792:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 411 00000100:00100000:0.0:1789302786.319014:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350384384:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 779us (1103us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302786.319028:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 408 00000100:00100000:3.0:1789302786.321367:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 414 00000100:00100000:0.0:1789302786.364599:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302786.364666:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350384896 00000100:00100000:1.0:1789302786.364680:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350384896 00000100:00100000:1.0:1789302786.364763:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1117 00000100:00100000:1.0:1789302786.364776:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:28680:x1876219350384896:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1789302786.365111:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:28680:x1876219350384896:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 337us (525us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302786.365121:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1117 00000100:00100000:2.0:1789302786.572805:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371133696:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302786.572918:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302786.576628:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371133696 00000100:00100000:1.0:1789302786.576650:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371133696 00000100:00100000:1.0:1789302786.576724:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:1.0:1789302786.576736:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371133696:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302786.578598:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 78184448 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302786.578721:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371133696:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1985us (5799us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302786.578732:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:2.0:1789302786.579742:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371133696:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302786.703074:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302787.084825:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302787.087513:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371133824:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302787.087624:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302787.090677:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371133824 00000100:00100000:2.0:1789302787.090702:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371133824 00000100:00100000:2.0:1789302787.091168:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 179 00000100:00100000:2.0:1789302787.091182:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371133824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302787.091233:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302787.091357:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371133824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 178us (3732us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302787.091365:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 179 00000100:00100000:3.0:1789302787.091697:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114bb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371133824:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1789302787.094451:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789302787.724614:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:3.0:1789302788.177404:0:20434:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12o: Layout-swap failure during Restore leaves file released ========================================================== 08:33:06 (1789302786) 00000100:00100000:1.0:1789302788.360590:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.360635:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350385024 00000100:00100000:1.0:1789302788.360648:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350385024 00000100:00100000:1.0:1789302788.360728:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1118 00000100:00100000:1.0:1789302788.360741:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:29346:x1876219350385024:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1789302788.360878:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:29346:x1876219350385024:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 139us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302788.360886:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1118 00000100:00080000:1.0:1789302788.360915:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302788.371368:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302788.371918:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350385152 00000100:00100000:3.0:1789302788.371932:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350385152 00000100:00100000:3.0:1789302788.372021:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1119 00000100:00100000:3.0:1789302788.372034:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:29347:x1876219350385152:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302788.372061:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302788.372088:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e2ee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0778 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.372185:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.372204:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.372219:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789302788.376089:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302788.377171:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377196:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377328:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377581:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377593:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377599:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377605:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302788.377609:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377616:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377629:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:3.0:1789302788.377631:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302788.377636:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377642:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e2f5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377654:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377659:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377665:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377670:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302788.377673:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377680:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377686:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff960f0cdf2d00) 00010000:00010000:3.0:1789302788.377689:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302788.377691:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377696:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e2fc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377741:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e2ee lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0778 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377752:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:3.0:1789302788.377753:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302788.377757:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e2ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0778 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.377765:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302788.377834:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:29347:x1876219350385152:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 5801us (6479us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302788.377842:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1119 00000100:00100000:1.0:1789302788.389767:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.390108:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350385408 00000100:00100000:0.0:1789302788.390122:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350385408 00000100:00100000:0.0:1789302788.390200:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1120 00000100:00100000:0.0:1789302788.390211:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:29347:x1876219350385408:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:0.0:1789302788.390288:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302788.390321:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e0f6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06e5 expref: 93 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.390339:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:0.0:1789302788.390345:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302788.390718:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.390735:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 42 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.390745:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.390784:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 42 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.390800:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 42 pid: 5844 timeout: 725 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302788.390827:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70c700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371133952:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302788.390954:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302788.399023:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302788.399223:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70c700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371133952:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302788.399260:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302788.406053:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350385536 00000100:00100000:3.0:1789302788.406074:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350385536 00010000:00010000:3.0:1789302788.406148:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 43 pid: 5844 timeout: 725 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302788.406200:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:3.0:1789302788.406210:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:2347:x1876219350385536:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302788.406230:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302788.406238:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 43 pid: 5844 timeout: 725 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.406253:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 43 pid: 5844 timeout: 725 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.406278:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e120 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a06f3 expref: 43 pid: 5844 timeout: 725 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.406364:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:3.0:1789302788.406371:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.406383:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.406391:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.406432:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302788.406657:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.406667:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.406698:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.406710:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.406718:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302788.407952:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 16384 00010000:00010000:2.0:1789302788.407967:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.407977:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.407984:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.407991:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302788.407996:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.408004:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.408031:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f02d72800) 00010000:00010000:2.0:1789302788.408033:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302788.408038:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.408044:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e303 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302788.408064:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d6d35f40 reply state 000000005450637c transno 4294968036 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302788.408073:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302788.408211:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:2347:x1876219350385536:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1997us (9198us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302788.408223:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:2.0:1789302788.411259:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:29347:x1876219350385408:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 21034us (21492us total) trans 4294968036 rc 0/0 00000100:00100000:2.0:1789302788.411274:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1120 00010000:00010000:2.0:1789302788.418746:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.418770:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.418779:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.418786:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302788.418793:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.418803:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.418848:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff960f11d36300) 00010000:00010000:2.0:1789302788.418851:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302788.418859:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.418865:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184e30a lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302788.421939:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302788.423522:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350385664 00000100:00100000:2.0:1789302788.423540:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350385664 00000100:00100000:2.0:1789302788.423672:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1121 00000100:00100000:2.0:1789302788.423684:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+93:29348:x1876219350385664:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302788.423716:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302788.423745:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e311 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a077f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.423830:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e318 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.423847:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e318 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.423862:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e318 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302788.423907:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302788.423912:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302788.423938:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e318 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.423991:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e311 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a077f expref: 95 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.424010:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e318 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a077f expref: 94 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.424018:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302788.424021:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302788.424026:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302788.424105:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:29348:x1876219350385664:12345-192.168.203.11@tcp:101:bash.0 Request processed in 423us (2182us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302788.424112:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1121 00000100:00100000:3.0:1789302788.429320:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.429630:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350385792 00000100:00100000:0.0:1789302788.429644:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350385792 00000100:00100000:0.0:1789302788.429702:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1122 00000100:00100000:0.0:1789302788.429712:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+94:29348:x1876219350385792:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302788.429738:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.429763:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e31f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0786 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.429845:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.429863:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.429877:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.429891:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.429898:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.429905:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430018:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e334 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430042:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e334 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430056:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e334 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302788.430089:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 16384 00000004:00200000:0.0:1789302788.430093:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302788.430112:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e334 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430127:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430144:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430152:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430159:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01662800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.430163:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430171:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430184:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302788.430187:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.430193:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430210:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e326 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430222:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430227:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430234:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430239:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.430242:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430247:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430254:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff960f03501f00) 00010000:00010000:0.0:1789302788.430256:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.430258:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430263:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e32d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430294:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e31f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0786 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430307:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eb800/0x64fe43095184e334 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0786 expref: 95 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.430316:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302788.430318:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302788.430320:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eb800, rc 0) 00000100:00100000:0.0:1789302788.430395:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29348:x1876219350385792:12345-192.168.203.11@tcp:101:bash.0 Request processed in 685us (1091us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302788.430402:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1122 00000100:00100000:0.0:1789302788.432596:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302788.432682:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350385920 00000100:00100000:2.0:1789302788.432694:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350385920 00000100:00100000:2.0:1789302788.432756:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1123 00000100:00100000:2.0:1789302788.432767:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29348:x1876219350385920:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302788.432791:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302788.432817:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e33b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a078d expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.432900:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.432916:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.432929:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.432964:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.432972:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.432979:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433066:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433074:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433084:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433091:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302788.433096:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433106:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433118:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302788.433122:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302788.433129:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433145:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e342 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433168:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433175:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433181:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433188:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302788.433191:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433197:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433206:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f03927400) 00010000:00010000:2.0:1789302788.433209:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302788.433212:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433217:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e349 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433255:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e33b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a078d expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433274:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302788.433276:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302788.433290:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e33b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a078d expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.433300:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302788.433371:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29348:x1876219350385920:12345-192.168.203.11@tcp:101:bash.0 Request processed in 606us (787us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302788.433379:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1123 00000100:00100000:2.0:1789302788.439892:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.439997:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350386048 00000100:00100000:1.0:1789302788.440010:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350386048 00000100:00100000:1.0:1789302788.440097:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1124 00000100:00100000:1.0:1789302788.440110:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+42:29349:x1876219350386048:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302788.440143:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302788.440168:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e350 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0794 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.440250:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.440285:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.440301:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302788.440349:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302788.440354:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302788.440378:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.440416:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e350 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 44 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.440437:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.440446:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302788.440450:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302788.440455:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6a00, rc 0) 00000100:00100000:1.0:1789302788.440575:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:29349:x1876219350386048:12345-192.168.203.11@tcp:101:rm.0 Request processed in 467us (698us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302788.440582:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1124 00000100:00100000:0.0:1789302788.443844:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.443934:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350386176 00000100:00100000:1.0:1789302788.443948:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350386176 00000100:00100000:1.0:1789302788.444019:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1125 00000100:00100000:1.0:1789302788.444031:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:29349:x1876219350386176:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302788.444058:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302788.444100:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e35e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a079b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444196:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444214:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444228:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444245:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444253:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444259:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444379:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e373 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444388:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e373 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444396:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e373 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302788.444428:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 16384 00000004:00200000:1.0:1789302788.444432:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302788.444454:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e373 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444476:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444483:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444491:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444498:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302788.444546:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444555:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444570:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302788.444573:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.444579:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444586:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e365 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444599:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444605:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444611:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444621:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302788.444625:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444631:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444639:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xa1420b2e (ffff960f10d24d00) 00010000:00010000:1.0:1789302788.444642:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.444644:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444650:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e36c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xa1420b2e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444686:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e35e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a079b expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444701:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7800/0x64fe43095184e373 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a079b expref: 44 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.444708:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302788.444711:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302788.444714:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7800, rc 0) 00000100:00100000:1.0:1789302788.444786:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29349:x1876219350386176:12345-192.168.203.11@tcp:101:rm.0 Request processed in 758us (958us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302788.444794:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1125 00000100:00100000:2.0:1789302788.447220:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.447311:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350386304 00000100:00100000:1.0:1789302788.447341:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350386304 00000100:00100000:1.0:1789302788.447427:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1126 00000100:00100000:1.0:1789302788.447439:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29349:x1876219350386304:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302788.447475:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302788.447537:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e37a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07a2 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447621:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447646:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447662:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447682:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447689:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447695:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447774:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447782:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447790:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447805:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302788.447810:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447819:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447834:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302788.447838:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.447843:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447850:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e381 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447866:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447872:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447879:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447884:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302788.447887:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447895:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447913:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f10d24000) 00010000:00010000:1.0:1789302788.447915:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.447919:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447925:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e388 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447971:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e37a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07a2 expref: 45 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447982:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302788.447985:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.447988:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e37a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07a2 expref: 45 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.447998:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302788.448070:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29349:x1876219350386304:12345-192.168.203.11@tcp:101:rm.0 Request processed in 634us (865us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302788.448078:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1126 00000100:00100000:3.0:1789302788.451647:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.451700:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350386432 00000100:00100000:0.0:1789302788.451714:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350386432 00000100:00100000:0.0:1789302788.451796:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1127 00000100:00100000:0.0:1789302788.451808:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29349:x1876219350386432:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302788.451837:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.451865:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e38f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07a9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.451949:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.451969:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.451983:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452001:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452015:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452023:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452090:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452099:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452106:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452113:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.452118:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452127:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452149:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302788.452153:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.452159:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452165:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e396 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452180:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452186:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452192:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452196:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.452199:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452205:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452213:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f03501f00) 00010000:00010000:0.0:1789302788.452215:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.452218:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452225:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e39d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452263:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e38f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07a9 expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452274:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302788.452277:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.452280:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e38f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07a9 expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.452295:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:0.0:1789302788.452376:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29349:x1876219350386432:12345-192.168.203.11@tcp:101:rm.0 Request processed in 570us (741us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302788.452383:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1127 00000100:00100000:2.0:1789302788.468790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.469553:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350386560 00000100:00100000:0.0:1789302788.469568:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350386560 00000100:00100000:0.0:1789302788.469700:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1128 00000100:00100000:0.0:1789302788.469714:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29350:x1876219350386560:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302788.469746:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.469774:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e3a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07b0 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.469865:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.469883:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.469897:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.469916:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.469923:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.469931:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.469999:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470007:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470015:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470023:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.470029:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470038:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470053:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302788.470056:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.470062:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470069:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e3ab lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470085:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470091:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470097:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470104:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.470107:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470113:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470120:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f03501f00) 00010000:00010000:0.0:1789302788.470123:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.470126:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470142:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e3b2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470185:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e3a4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07b0 expref: 45 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470197:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302788.470200:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.470204:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e3a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07b0 expref: 45 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.470215:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:0.0:1789302788.470304:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29350:x1876219350386560:12345-192.168.203.11@tcp:101:cp.0 Request processed in 593us (1533us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302788.470311:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1128 00000100:00100000:2.0:1789302788.473226:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.473386:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350386688 00000100:00100000:1.0:1789302788.473399:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350386688 00000100:00100000:1.0:1789302788.473479:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1129 00000100:00100000:1.0:1789302788.473492:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29350:x1876219350386688:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302788.473555:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302788.473583:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e3b9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07b7 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473692:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473712:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473726:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473743:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473751:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473759:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473850:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473859:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473867:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473874:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302788.473880:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473901:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473916:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302788.473920:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.473927:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473933:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e3c0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473949:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473954:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473960:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473966:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302788.473968:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473974:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.473981:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f10d24000) 00010000:00010000:1.0:1789302788.473983:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.473986:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.474001:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e3c7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.474047:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e3b9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07b7 expref: 45 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.474058:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302788.474060:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.474064:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e3b9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07b7 expref: 45 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.474074:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302788.474164:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29350:x1876219350386688:12345-192.168.203.11@tcp:101:cp.0 Request processed in 673us (952us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302788.474171:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1129 00000100:00100000:3.0:1789302788.480623:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.480768:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350386944 00000100:00100000:1.0:1789302788.480782:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350386944 00000100:00100000:1.0:1789302788.480863:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1130 00000100:00100000:1.0:1789302788.480877:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29350:x1876219350386944:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302788.480906:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302788.480934:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e3ce lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07be expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.481149:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.481168:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.481192:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.481210:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 3/0,1 mode: --/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.481218:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.481225:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302788.482057:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x17:0x0], index 1: 0 00010000:00010000:1.0:1789302788.482296:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482310:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482320:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482362:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482370:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482378:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482385:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302788.482399:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482409:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482425:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302788.482429:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302788.482435:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482443:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e3d5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302788.482459:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000935725c0 reply state 000000008d9612d3 transno 4294968037 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302788.482469:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302788.482544:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e3ce lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 47 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482565:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.482572:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302788.482575:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302788.482577:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dda00, rc 0) 00000100:00100000:1.0:1789302788.482678:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29350:x1876219350386944:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1803us (2075us total) trans 4294968037 rc 0/0 00000100:00100000:1.0:1789302788.482686:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1130 00010000:00010000:3.0:1789302788.488494:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.488615:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.488625:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.488632:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302788.488638:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.488647:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.488670:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f10d24d00) 00010000:00010000:3.0:1789302788.488674:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302788.488680:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 2/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.488687:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e3dc lrc: 0/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302788.490167:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.490477:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350387072 00000100:00100000:0.0:1789302788.490491:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350387072 00000100:00080000:1.0:1789302788.490559:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302788.490607:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 302 00000100:00100000:0.0:1789302788.490625:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350387072:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302788.490650:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.490684:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07c5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302788.490720:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x17:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302788.490746:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=625s v=(0 0) 00010000:00010000:0.0:1789302788.490751:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302788.490762:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.490786:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.490807:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ebc00, rc 0) 00000100:00100000:0.0:1789302788.490889:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350387072:12345-192.168.203.11@tcp:101:cp.0 Request processed in 266us (733us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302788.490897:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 302 00000040:00080000:3.0:1789302788.492741:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302788.492752:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302788.493258:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1789302788.493268:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302788.494565:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.494686:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350387200 00000100:00100000:0.0:1789302788.494699:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350387200 00000100:00100000:0.0:1789302788.494767:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 303 00000100:00100000:0.0:1789302788.494779:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:29350:x1876219350387200:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302788.494811:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.494836:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07cc expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.494876:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.494887:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302788.494891:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302788.494897:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.494939:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.494955:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 11 pid: 6697 timeout: 725 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302788.494971:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371134080:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302788.495131:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302788.501271:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302788.501380:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350387328 00000100:00100000:3.0:1789302788.501393:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350387328 00010000:00010000:3.0:1789302788.501452:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 12 pid: 6697 timeout: 725 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302788.501539:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:3.0:1789302788.501552:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350387328:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302788.501572:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302788.501603:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 12 pid: 6697 timeout: 725 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.501621:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 12 pid: 6697 timeout: 725 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.501659:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302788.501665:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302788.501676:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.501687:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 12 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.501718:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e3f1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07c5 expref: 12 pid: 6697 timeout: 725 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302788.501734:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302788.501735:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371134080:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302788.501752:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b0aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371134208:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302788.501766:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.501776:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302788.501832:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:29350:x1876219350387200:12345-192.168.203.11@tcp:101:cp.0 Request processed in 7054us (7277us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302788.501840:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 303 00000100:00100000:3.0:1789302788.502003:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f095bfd80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302788.505001:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302788.505127:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b0aa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371134208:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302788.505173:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302788.505292:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350387328:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 3740us (4038us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302788.505308:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:0.0:1789302788.505407:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350387584 00000100:00100000:0.0:1789302788.505416:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350387584 00000100:00100000:0.0:1789302788.505485:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 224 00000100:00100000:0.0:1789302788.505496:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29350:x1876219350387584:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302788.505948:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29350:x1876219350387584:12345-192.168.203.11@tcp:35:cp.0 Request processed in 454us (967us total) trans 4294968038 rc 0/0 00000100:00100000:0.0:1789302788.505957:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 224 00000100:00100000:2.0:1789302788.537066:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.537154:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350387712 00000100:00100000:0.0:1789302788.537315:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350387712 00000100:00100000:0.0:1789302788.537397:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1131 00000100:00100000:0.0:1789302788.537410:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29353:x1876219350387712:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302788.537438:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.537468:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e3ff lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07d3 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537619:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537637:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537650:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537676:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537684:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537700:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537809:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537818:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537825:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302788.537868:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:0.0:1789302788.537874:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302788.537897:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537911:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537920:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537926:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537933:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537943:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.537948:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537956:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537968:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302788.537972:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.537978:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.537985:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e406 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538000:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538007:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538013:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538018:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302788.538021:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538027:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538039:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f03501100) 00010000:00010000:0.0:1789302788.538042:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302788.538045:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538050:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e40d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538086:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e3ff lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 97 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538098:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.538116:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302788.538118:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302788.538122:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01663800, rc 0) 00000100:00100000:0.0:1789302788.538213:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29353:x1876219350387712:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 805us (1165us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302788.538221:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1131 00000100:00100000:2.0:1789302788.545483:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.545683:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350387840 00000100:00100000:0.0:1789302788.545696:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350387840 00000100:00100000:0.0:1789302788.545796:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 304 00000100:00100000:0.0:1789302788.545816:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350387840:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302788.545840:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.545865:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e41b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07da expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302788.545948:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371134336:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302788.546063:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3480 going to sleep for 11 seconds 00000100:00080000:1.0:1789302788.546711:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:0.0:1789302788.548349:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1789302788.548359:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb mtime: 0 -> 1789302788 00002000:00010000:0.0:1789302788.548362:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb atime: 0 -> 1789302788 00002000:00010000:0.0:1789302788.548365:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb ctime: 0 -> 1789302788 00000100:00100000:0.0:1789302788.548399:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371134336:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302788.548444:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e41b lrc: 1/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07da expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.548472:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e41b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07da expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302788.548486:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01662800, rc 0) 00000100:00100000:0.0:1789302788.548581:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350387840:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2768us (3113us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302788.548591:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 304 00000100:00100000:2.0:1789302788.553265:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.553642:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350387968 00000100:00100000:1.0:1789302788.553656:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350387968 00000100:00100000:1.0:1789302788.553724:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1132 00000100:00100000:1.0:1789302788.553737:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29353:x1876219350387968:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302788.553765:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302788.553810:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184e422 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07e1 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.553912:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.553942:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.553958:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302788.554011:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x46:0x0]: returning size 16384 00000004:00200000:1.0:1789302788.554017:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302788.554039:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.554074:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe43095184e422 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 98 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.554097:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 97 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302788.554129:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302788.554144:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302788.554149:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:1.0:1789302788.554433:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:29353:x1876219350387968:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 697us (1189us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302788.554441:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1132 00000100:00100000:2.0:1789302788.569797:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302788.570400:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350388096 00000100:00100000:0.0:1789302788.570417:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350388096 00000100:00100000:0.0:1789302788.570608:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:0.0:1789302788.570621:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350388096:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302788.570645:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302788.570669:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e430 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07e8 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302788.570756:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371134464:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302788.571161:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302788.574835:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371134464:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302788.574889:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e430 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e8 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.574920:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e430 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e8 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302788.574935:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01662800, rc 0) 00000100:00100000:2.0:1789302788.575028:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350388096:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4410us (5253us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302788.575040:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:3.0:1789302788.588487:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302788.588962:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350388352 00000100:00100000:1.0:1789302788.588976:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350388352 00000100:00100000:1.0:1789302788.589051:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1133 00000100:00100000:1.0:1789302788.589064:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:29353:x1876219350388352:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:1.0:1789302788.589471:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].31, 136 off12408 00000100:00100000:1.0:1789302788.589635:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:29353:x1876219350388352:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 572us (1161us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302788.589644:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1133 00010000:00010000:2.0:1789302788.749852:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302789.477376:0:20484:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302789.477388:0:20484:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302789.477445:0:20484:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 32 count 5 00000040:00080000:2.0:1789302789.477451:0:20484:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789302789.517054:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302789.517068:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302789.517415:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 32 count 3 00000040:00080000:3.0:1789302789.517423:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302789.518632:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371134592:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302789.518766:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302789.522625:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371134592:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302789.529749:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302789.532333:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350388480 00000100:00100000:0.0:1789302789.532354:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350388480 00000100:00100000:0.0:1789302789.532437:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1134 00000100:00100000:0.0:1789302789.532450:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29365:x1876219350388480:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302789.532499:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302789.532669:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29365:x1876219350388480:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 221us (2936us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302789.532679:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1134 00000100:00100000:3.0:1789302789.537605:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302789.537718:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350388736 00000100:00100000:1.0:1789302789.537732:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350388736 00000100:00100000:1.0:1789302789.537800:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1135 00000100:00100000:1.0:1789302789.537813:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29365:x1876219350388736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302789.537842:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302789.537871:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e437 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07ef expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.538001:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e43e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.538018:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e43e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.538047:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e43e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302789.538061:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:1.0:1789302789.538252:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e437 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07ef expref: 50 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.538275:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e43e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07ef expref: 49 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.538286:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302789.538290:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302789.538294:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302789.538376:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29365:x1876219350388736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 566us (785us total) trans 4294968039 rc 0/0 00000100:00100000:1.0:1789302789.538385:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1135 00000100:00100000:0.0:1789302789.543058:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302789.543102:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302789.543144:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350388992 00000100:00100000:0.0:1789302789.543158:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350388992 00000100:00100000:0.0:1789302789.543236:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 49 00000100:00100000:0.0:1789302789.543247:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350388992:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302789.543267:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302789.543295:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e43e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07ef expref: 49 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302789.543315:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302789.543321:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302789.543376:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2347:x1876219350388992:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 131us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302789.543384:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 49 00000100:00100000:3.0:1789302789.543835:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350388864 00000100:00100000:3.0:1789302789.543844:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350388864 00000100:00100000:3.0:1789302789.543898:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:3.0:1789302789.543906:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350388864:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302789.543953:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.543977:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.543987:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x17:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302789.543994:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302789.543998:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 4/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.544040:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.544057:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 10 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302789.544071:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371134720:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302789.544153:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:3.0:1789302789.549754:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302789.549798:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350389120 00000100:00100000:3.0:1789302789.549808:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350389120 00002000:00100000:3.0:1789302789.549868:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x17:0x0] (0->157) req@ffff960e02b0ad80 x1876219350389120/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:365/0 lens 648/0 e 0 to 0 dl 1789302800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302789.549880:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02b0ad80 with x1876219350389120 ext(0->157) 00010000:00010000:3.0:1789302789.549892:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 11 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.549911:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 11 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302789.549924:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960e02b0ad80 00000100:00100000:3.0:1789302789.549960:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 10 00000100:00100000:3.0:1789302789.549971:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350389120:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302789.550493:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371134720:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302789.550553:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302789.551027:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350389120:12345-192.168.203.11@tcp:4:cp.0 Request processed in 1056us (1285us total) trans 4294967360 rc 0/0 00000100:00100000:3.0:1789302789.551038:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 10 00002000:00100000:3.0:1789302789.551132:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x17:0x0] (0->157) req@ffff960e02b0ad80 x1876219350389120/t4294967360(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:365/0 lens 648/448 e 0 to 0 dl 1789302800 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302789.551144:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02b0ad80 with x1876219350389120 ext(0->157) 00010000:00010000:3.0:1789302789.551151:0:6701:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 13 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.551166:0:6701:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 13 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302789.551179:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960e02b0ad80 00000100:00100000:3.0:1789302789.553948:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302789.554065:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350389248 00000100:00100000:3.0:1789302789.554077:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350389248 00010000:00010000:3.0:1789302789.554138:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 13 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302789.554189:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:3.0:1789302789.554200:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2346:x1876219350389248:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302789.554218:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302789.554247:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 13 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.554269:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 13 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.554299:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003501a982/0x64fe43095184e3f8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07cc expref: 13 pid: 8571 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.554318:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302789.554323:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302789.554329:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.554338:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555550:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302789.555635:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350389248:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1431us (1695us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302789.555646:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 339 00010000:00010000:3.0:1789302789.555822:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555838:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555876:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555886:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555895:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555903:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302789.555908:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555919:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555938:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.555946:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184e44c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302789.556013:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350388864:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 12107us (12969us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302789.556022:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:3.0:1789302789.560683:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302789.560776:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350389504 00000100:00100000:2.0:1789302789.560790:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350389504 00000100:00100000:2.0:1789302789.560865:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1136 00000100:00100000:2.0:1789302789.560877:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29365:x1876219350389504:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302789.561023:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29365:x1876219350389504:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 148us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302789.561031:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1136 00000100:00100000:1.0:1789302789.571180:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302789.571295:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350389632 00000100:00100000:3.0:1789302789.571308:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350389632 00000100:00100000:3.0:1789302789.571380:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 307 00000100:00100000:3.0:1789302789.571391:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350389632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302789.571418:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302789.571446:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07f6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302789.571478:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x17:0x0] initial LVB size: 158, mtime: 0x6aa69804, atime: 0x0, ctime: 0x6aa69804, blocks: 0x1 00010000:00010000:3.0:1789302789.571521:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=626s v=(0 0) 00010000:00010000:3.0:1789302789.571526:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302789.571537:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.571559:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.571574:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302789.571646:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350389632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 256us (478us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302789.571653:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 307 00000100:00100000:1.0:1789302789.575741:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302789.575833:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350389888 00000100:00100000:2.0:1789302789.575848:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350389888 00000100:00100000:2.0:1789302789.575916:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1137 00000100:00100000:2.0:1789302789.575927:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29365:x1876219350389888:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302789.576068:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29365:x1876219350389888:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 143us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302789.576077:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1137 00000100:00100000:3.0:1789302789.582174:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302789.582290:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350390016 00000100:00100000:1.0:1789302789.582304:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350390016 00002000:00100000:1.0:1789302789.582383:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x17:0x0] (0->4095) req@ffff960e02b08000 x1876219350390016/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:365/0 lens 488/0 e 0 to 0 dl 1789302800 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302789.582414:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02b08000 with x1876219350390016 ext(0->4095) 00002000:00010000:1.0:1789302789.582420:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e02b08000 00000100:00100000:1.0:1789302789.582462:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 103 00000100:00100000:1.0:1789302789.582474:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350390016:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302789.582753:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350390016:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 280us (592us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302789.582761:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 103 00002000:00100000:1.0:1789302789.582768:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x17:0x0] (0->4095) req@ffff960e02b08000 x1876219350390016/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:365/0 lens 488/4536 e 0 to 0 dl 1789302800 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302789.582777:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02b08000 with x1876219350390016 ext(0->4095) 00002000:00010000:1.0:1789302789.582780:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e02b08000 00000100:00100000:3.0:1789302789.585725:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302789.586024:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350390144 00000100:00100000:1.0:1789302789.586037:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350390144 00000100:00100000:1.0:1789302789.586103:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1138 00000100:00100000:1.0:1789302789.586116:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29365:x1876219350390144:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302789.586151:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302789.586180:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e45a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a07fd expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.586236:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.586254:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.586287:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.586529:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e45a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 50 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.586548:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.586558:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302789.586561:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302789.586566:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302789.586641:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29365:x1876219350390144:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 528us (930us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302789.586649:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1138 00000100:00100000:1.0:1789302789.590815:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302789.590935:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350390272 00000100:00100000:3.0:1789302789.590948:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350390272 00000100:00100000:3.0:1789302789.591022:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1139 00000100:00100000:3.0:1789302789.591034:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29365:x1876219350390272:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302789.591078:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302789.591339:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29365:x1876219350390272:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 306us (538us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302789.591349:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1139 00000100:00100000:0.0:1789302789.598914:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302789.599023:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350390400 00000100:00100000:1.0:1789302789.599037:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350390400 00000100:00100000:1.0:1789302789.599110:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 308 00000100:00100000:1.0:1789302789.599122:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350390400:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302789.599206:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.599227:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302789.599232:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302789.599239:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.599273:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.599313:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.599322:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.599331:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.599339:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302789.599345:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.600592:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.600855:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.600866:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184e468 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302789.601312:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350390400:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 2184us (2403us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302789.601324:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 308 00000100:00100000:1.0:1789302789.612565:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302789.612658:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350390656 00000100:00100000:2.0:1789302789.612672:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350390656 00000100:00100000:2.0:1789302789.612749:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1140 00000100:00100000:2.0:1789302789.612771:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29365:x1876219350390656:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302789.612880:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302789.612902:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302789.612911:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 4/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302789.612954:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302789.612968:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 49 pid: 5844 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302789.612985:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371134848:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302789.613098:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c439c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302789.618632:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371134848:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302789.618673:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302789.619165:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302789.619379:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350390784 00000100:00100000:3.0:1789302789.619389:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350390784 00010000:00010000:3.0:1789302789.619438:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 50 pid: 5844 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302789.619476:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:3.0:1789302789.619488:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2348:x1876219350390784:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302789.619546:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302789.619554:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 50 pid: 5844 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.619570:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 50 pid: 5844 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.619597:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e461 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07fd expref: 50 pid: 5844 timeout: 726 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.619617:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302789.619623:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.619639:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.619649:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302789.619680:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302789.619746:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350390784:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 260us (584us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302789.619754:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 340 00010000:00010000:1.0:1789302789.619987:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.619997:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.620290:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.620298:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.620306:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.620313:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302789.620318:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.620328:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.620340:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302789.620343:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302789.620378:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 2/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302789.620385:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e46f lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302789.620584:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29365:x1876219350390656:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 7815us (8034us total) trans 4294968040 rc 0/0 00000100:00100000:1.0:1789302789.620595:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1140 00000040:00080000:3.0:1789302789.621118:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302789.621139:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302789.621723:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:3.0:1789302789.621734:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302789.629051:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302789.629143:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391040 00000100:00100000:3.0:1789302789.629156:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391040 00000100:00100000:3.0:1789302789.629234:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 225 00000100:00100000:3.0:1789302789.629248:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29365:x1876219350391040:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302789.629754:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29365:x1876219350391040:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 508us (715us total) trans 4294968041 rc 0/0 00000100:00100000:3.0:1789302789.629763:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 225 00010000:00010000:1.0:1789302789.775617:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302790.796892:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789302790.814608:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967360 is committed 00000001:00080000:3.0:1789302790.824412:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968038 is committed 00000001:00080000:3.0:1789302790.824440:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968040 is committed 00000020:00080000:3.0:1789302790.824455:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 22 batchid updated 00000001:00080000:3.0:1789302790.824476:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968036 is committed 00000001:00080000:3.0:1789302790.824486:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968041 is committed 00000020:00080000:3.0:1789302790.824528:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 22 00000020:00080000:3.0:1789302790.824542:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 22 00000100:00100000:0.0:1789302791.182158:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371134976:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.182264:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302791.182324:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371134976 00000100:00100000:3.0:1789302791.182337:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371134976 00000100:00100000:0.0:1789302791.182341:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371135488:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.182381:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302791.182412:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302791.182418:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371135360:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.182449:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302791.182480:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b08700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371135104:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.182534:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302791.182561:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371135616:0@lo:400:kworker.0 00000100:00100000:3.0:1789302791.182584:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 570 00000100:00100000:0.0:1789302791.182593:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302791.182601:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371134976:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.182621:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302791.182630:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371135232:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.182668:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302791.182688:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371134976:12345-0@lo:400:kworker.0 Request processed in 91us (422us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302791.182716:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371135360 00000100:00100000:0.0:1789302791.182722:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371135360 00000100:00100000:0.0:1789302791.182774:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371135616 00000100:00100000:0.0:1789302791.182777:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371135616 00000100:00100000:3.0:1789302791.182795:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 570 00000100:00100000:0.0:1789302791.182815:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1789302791.182826:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371135360:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.182873:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371135360:12345-0@lo:400:kworker.0 Request processed in 50us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302791.182881:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:0.0:1789302791.182898:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1789302791.182908:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371135616:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302791.182922:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371135488 00000100:00100000:2.0:1789302791.182928:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371135488 00000100:00100000:0.0:1789302791.182955:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371135616:12345-0@lo:400:kworker.0 Request processed in 48us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302791.182963:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1789302791.182969:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371135104 00000100:00100000:2.0:1789302791.182973:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371135104 00000100:00100000:2.0:1789302791.182994:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371135232 00000100:00100000:2.0:1789302791.182997:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371135232 00000100:00100000:2.0:1789302791.183013:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789302791.183020:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371135488:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302791.183058:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371135488:12345-0@lo:400:kworker.0 Request processed in 39us (677us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302791.183074:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1789302791.183089:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:2.0:1789302791.183095:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371135104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302791.183151:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371135104:12345-0@lo:400:kworker.0 Request processed in 55us (617us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302791.183157:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:3.0:1789302791.183163:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371135616:0@lo:400:kworker.0 00000100:00100000:2.0:1789302791.183165:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:0.0:1789302791.183172:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371135232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302791.183200:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b08700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371135104:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.183298:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371135232:12345-0@lo:400:kworker.0 Request processed in 132us (629us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302791.183306:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00080000:0.0:1789302791.183331:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302791.183337:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302791.183342:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302791.183373:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967360 gen 2 00000004:00080000:0.0:1789302791.183380:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e0114a680, transno 4294967360 00000100:00100000:0.0:1789302791.183391:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371135360:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.183403:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b0a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371134976:0@lo:400:kworker.0 00000100:00100000:0.0:1789302791.183415:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371135488:0@lo:400:kworker.0 00000100:00100000:1.0:1789302791.185248:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371135232:0@lo:400:kworker.0 00000001:00080000:2.0:1789302791.247414:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967360 is committed 00000100:00100000:2.0:1789302791.444404:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.444467:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.444530:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.444554:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.444580:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.444612:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.444642:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.444698:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391552 00000100:00100000:2.0:1789302791.444711:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391552 00000100:00100000:3.0:1789302791.444862:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391296 00000100:00100000:3.0:1789302791.444867:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391296 00000100:00100000:1.0:1789302791.444927:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391168 00000100:00100000:1.0:1789302791.444932:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391168 00000100:00100000:1.0:1789302791.444989:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 571 00000100:00100000:1.0:1789302791.445001:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350391168:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302791.445089:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350391168:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 90us (697us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302791.445097:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 571 00000100:00100000:2.0:1789302791.447652:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391424 00000100:00100000:2.0:1789302791.447670:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391424 00000100:00100000:2.0:1789302791.447705:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391808 00000100:00100000:2.0:1789302791.447708:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391808 00000100:00100000:2.0:1789302791.447721:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391936 00000100:00100000:2.0:1789302791.447723:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391936 00000100:00100000:2.0:1789302791.447768:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 414 00000100:00100000:2.0:1789302791.447778:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350391552:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302791.447868:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350391552:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 91us (3365us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302791.447875:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 414 00000100:00100000:2.0:1789302791.447900:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 415 00000100:00100000:2.0:1789302791.447906:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350391424:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302791.447939:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350391424:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (3385us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302791.447945:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 415 00000100:00100000:2.0:1789302791.447956:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 416 00000100:00100000:2.0:1789302791.447963:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350391808:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302791.447985:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350391808:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (3404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302791.447991:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 416 00000100:00100000:2.0:1789302791.448000:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 417 00000100:00100000:2.0:1789302791.448006:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350391936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302791.448029:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350391936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (3387us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302791.448034:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 417 00000100:00100000:2.0:1789302791.448072:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350391680 00000100:00100000:2.0:1789302791.448076:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350391680 00000100:00100000:2.0:1789302791.448108:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 418 00000100:00100000:2.0:1789302791.448130:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2345:x1876219350391680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302791.448167:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2345:x1876219350391680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 53us (3555us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302791.448174:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 418 00000100:00080000:0.0:1789302791.449847:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302791.449859:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302791.449862:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302791.451830:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 419 00000100:00100000:3.0:1789302791.451847:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2346:x1876219350391296:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302791.451948:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2346:x1876219350391296:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 106us (7479us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302791.451955:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 419 00000040:00080000:3.0:1789302791.507299:0:20539:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302791.507312:0:20539:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302791.507369:0:20539:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 32 count 3 00000040:00080000:3.0:1789302791.507377:0:20539:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302791.581777:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302791.582186:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350392064 00000100:00100000:0.0:1789302791.582200:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350392064 00000100:00100000:0.0:1789302791.582275:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 309 00000100:00100000:0.0:1789302791.582287:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350392064:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302791.582312:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302791.582338:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0804 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.582374:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=2s v=(0 0) 00010000:00010000:0.0:1789302791.582379:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302791.582390:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.582415:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.582430:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01662800, rc 0) 00000100:00100000:0.0:1789302791.582528:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350392064:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 242us (764us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302791.582535:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 309 00000100:00100000:0.0:1789302791.591057:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302791.591138:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350392320 00000100:00100000:0.0:1789302791.591151:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350392320 00000100:00100000:0.0:1789302791.591223:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1141 00000100:00100000:0.0:1789302791.591236:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29373:x1876219350392320:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302791.591266:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302791.591298:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e47d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a080b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.591437:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e484 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.591468:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e484 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.591485:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e484 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302791.591498:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1789302791.591818:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e47d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a080b expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.591839:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e484 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a080b expref: 98 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302791.591850:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302791.591853:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302791.591858:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302791.591943:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29373:x1876219350392320:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 709us (900us total) trans 4294968042 rc 0/0 00000100:00100000:0.0:1789302791.591951:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1141 00000100:00100000:1.0:1789302791.599644:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.600651:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350392448 00000100:00100000:2.0:1789302791.600668:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350392448 00000100:00100000:2.0:1789302791.600747:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 310 00000100:00100000:2.0:1789302791.600759:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350392448:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:2.0:1789302791.600818:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.600844:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.600854:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.600863:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302791.600868:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302791.600873:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.600917:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 17 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.600933:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 17 pid: 6697 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302791.600949:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371135744:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302791.601083:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 9 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.601107:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 9 pid: 6696 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302791.601132:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371135872:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302791.601162:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b180 going to sleep for 11 seconds 00000100:00100000:2.0:1789302791.619998:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266ca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371135744:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302791.620019:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371135872:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302791.620057:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302791.624940:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302791.625053:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302791.625495:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350392576 00000100:00100000:3.0:1789302791.625534:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350392576 00010000:00010000:3.0:1789302791.625595:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 18 pid: 6697 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302791.625650:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350392704 00000100:00100000:3.0:1789302791.625654:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350392704 00010000:00010000:3.0:1789302791.625670:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 10 pid: 6696 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302791.625696:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:3.0:1789302791.625706:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09752050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350392576:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302791.625725:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1789302791.625752:0:5830:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x17:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:3.0:1789302791.625760:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 18 pid: 6697 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.625775:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 18 pid: 6697 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.625803:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002ffb05b5/0x64fe43095184e476 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0804 expref: 18 pid: 6697 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.625821:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302791.625896:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09752050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350392576:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 191us (974us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302791.625904:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:3.0:1789302791.625988:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 342 00000100:00100000:3.0:1789302791.625995:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350392704:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302791.626006:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302791.626016:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 10 pid: 6696 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626026:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 10 pid: 6696 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626041:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e453 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x17:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07f6 expref: 10 pid: 6696 timeout: 728 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626053:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302791.626058:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302791.626065:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626076:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626130:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302791.626152:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350392704:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 157us (1098us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302791.626159:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 342 00010000:00010000:3.0:1789302791.626334:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626345:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626374:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626384:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626393:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626400:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302791.626405:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626415:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626427:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.626434:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006b24994b/0x64fe43095184e492 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302791.626491:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350392448:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 25735us (26860us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302791.626525:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 310 00000100:00100000:1.0:1789302791.633597:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302791.637583:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350392960 00000100:00100000:2.0:1789302791.637605:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350392960 00000100:00100000:2.0:1789302791.637692:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 226 00000100:00100000:2.0:1789302791.637705:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29373:x1876219350392960:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:2.0:1789302791.637789:0:5846:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x20:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e484 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a080b expref: 98 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302791.638107:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968043 00000001:00080000:2.0:1789302791.640284:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968044 00010000:00010000:2.0:1789302791.640417:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.640443:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 44 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.640472:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.640482:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.640553:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.640574:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 101 pid: 18427 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302791.640594:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371136000:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302791.642752:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 44 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.642868:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 44 pid: 5844 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302791.642885:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266ca80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371136128:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302791.642974:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b180 going to sleep for 11 seconds 00000100:00100000:0.0:1789302791.647367:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302791.647557:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350393088 00000100:00100000:3.0:1789302791.647569:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350393088 00010000:00010000:3.0:1789302791.647676:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 102 pid: 18427 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302791.647719:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 343 00000100:00100000:3.0:1789302791.647733:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:2347:x1876219350393088:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302791.647754:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302791.647762:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 102 pid: 18427 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.647777:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 102 pid: 18427 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.647804:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302791.647811:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.647822:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.647829:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302791.647893:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:2347:x1876219350393088:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 164us (541us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302791.647901:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 343 00010000:00010000:2.0:1789302791.648283:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e414 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07d3 expref: 101 pid: 18427 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302791.648305:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371136000:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302791.648331:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266ca80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371136128:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302791.648362:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302791.650386:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302791.650474:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350393216 00000100:00100000:3.0:1789302791.650489:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350393216 00010000:00010000:3.0:1789302791.650578:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 45 pid: 5844 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302791.650623:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 344 00000100:00100000:3.0:1789302791.650809:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2347:x1876219350393216:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302791.650829:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302791.650838:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 45 pid: 5844 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.650853:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 45 pid: 5844 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.650878:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095184e3e3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07be expref: 45 pid: 5844 timeout: 728 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.650896:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302791.650902:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.650914:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.650922:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302791.650956:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302791.651064:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.651073:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302791.651242:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968045 00000001:00080000:2.0:1789302791.651492:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968046 00000001:00080000:2.0:1789302791.651572:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968047 00000100:00100000:2.0:1789302791.651685:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371136256:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302791.651747:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302791.651783:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9ba80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302791.651811:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371136256 00000100:00100000:2.0:1789302791.651818:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371136256 00000100:00100000:2.0:1789302791.651859:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:2.0:1789302791.651867:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5846:x1876219371136256:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302791.651945:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5846:x1876219371136256:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 78us (196us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302791.651952:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:2.0:1789302791.652001:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371136256:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1789302791.652369:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968048 00000100:00100000:2.0:1789302791.652418:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371136384:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302791.652453:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302791.652469:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b180 going to sleep for 11 seconds 00000100:00100000:2.0:1789302791.652481:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371136384 00000100:00100000:2.0:1789302791.652485:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371136384 00000100:00100000:2.0:1789302791.652532:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:2.0:1789302791.652540:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5846:x1876219371136384:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302791.652758:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5846:x1876219371136384:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 217us (303us total) trans 4294967361 rc 0/0 00000100:00100000:2.0:1789302791.652764:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00000100:00100000:2.0:1789302791.652798:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266c380 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371136384:0@lo:1000:mdt_rdpg00_000.0 00010000:00010000:2.0:1789302791.652816:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.652826:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.652834:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.652841:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302791.652846:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.652855:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.652870:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:2.0:1789302791.652873:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1789302791.652880:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302791.652893:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e499 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302791.653074:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x47:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1789302791.653318:0:5846:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].10, 64 off8904 00000004:00080000:2.0:1789302791.653330:0:5846:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 628663827748ns, next at 629663825249ns, rc = 0 00000001:00080000:2.0:1789302791.653337:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968049 00010000:00010000:2.0:1789302791.653374:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:2.0:1789302791.653377:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1789302791.653385:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e484 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a080b expref: 105 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302791.653539:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350393216:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 2904us (3164us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302791.653550:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 344 00000100:00100000:2.0:1789302791.655793:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:29373:x1876219350392960:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 18074us (22193us total) trans 4294968050 rc 0/0 00000100:00100000:2.0:1789302791.655811:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 226 00000020:00080000:2.0:1789302791.655821:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:3.0:1789302791.697210:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302791.697245:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371136512:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302791.697348:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302791.697996:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371136512 00000100:00100000:3.0:1789302791.698008:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371136512 00000100:00100000:3.0:1789302791.698080:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:3.0:1789302791.698093:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371136512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302791.698151:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302791.698225:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038da300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371136512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 134us (878us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302791.698232:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00100000:1.0:1789302791.698332:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371136512:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302791.821767:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302792.207240:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371136640:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302792.207338:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302792.207945:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371136640 00000100:00100000:1.0:1789302792.207958:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371136640 00000100:00100000:1.0:1789302792.208226:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 181 00000100:00100000:1.0:1789302792.208238:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371136640:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302792.208304:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302792.208377:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371136640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 141us (1038us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302792.208385:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 181 00000100:00100000:0.0:1789302792.208487:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371136640:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302792.845641:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302793.707968:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302793.708039:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350393344 00000100:00100000:1.0:1789302793.708048:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350393344 00000100:00100000:1.0:1789302793.708127:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1142 00000100:00100000:1.0:1789302793.708139:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:29391:x1876219350393344:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:1.0:1789302793.708162:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302793.708192:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e4a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0812 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708253:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708267:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708280:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708294:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708301:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708308:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708688:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e4b5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708697:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e4b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708705:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e4b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302793.708739:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:1.0:1789302793.708744:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302793.708761:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e4b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708773:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e4b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708781:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708787:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708794:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708800:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302793.708804:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708812:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708829:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:1.0:1789302793.708832:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302793.708838:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708845:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e4a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708856:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708862:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708868:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708873:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302793.708876:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708881:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708887:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f10d24000) 00010000:00010000:1.0:1789302793.708890:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302793.708892:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708898:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e4ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708937:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e4a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0812 expref: 106 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708950:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e4b5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0812 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.708958:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:1.0:1789302793.708960:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302793.708962:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6e00, rc 0) 00000100:00100000:1.0:1789302793.709024:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:29391:x1876219350393344:12345-192.168.203.11@tcp:101:diff.0 Request processed in 888us (1074us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302793.709031:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1142 00000100:00080000:3.0:1789302793.709671:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302793.715887:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.716033:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350393600 00000100:00100000:3.0:1789302793.716043:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350393600 00000100:00100000:3.0:1789302793.716494:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1143 00000100:00100000:3.0:1789302793.716533:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:29391:x1876219350393600:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302793.716556:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302793.716577:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0819 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.716667:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4c3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.716682:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4c3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.716692:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4c3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302793.716708:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:3.0:1789302793.716821:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0819 expref: 108 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.716837:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4c3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0819 expref: 107 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.716845:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:3.0:1789302793.716848:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302793.716852:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302793.716910:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:29391:x1876219350393600:12345-192.168.203.11@tcp:101:diff.0 Request processed in 382us (1031us total) trans 4294968051 rc 0/0 00000100:00100000:3.0:1789302793.716917:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1143 00000100:00100000:2.0:1789302793.723034:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.723148:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350393856 00000100:00100000:3.0:1789302793.723158:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350393856 00000100:00100000:3.0:1789302793.723223:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 50 00000100:00100000:3.0:1789302793.723233:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:29391:x1876219350393856:12345-192.168.203.11@tcp:103:diff.0 00010000:00010000:3.0:1789302793.723248:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789302793.723271:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4c3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0819 expref: 107 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.723292:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e4b5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0812 expref: 106 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.723302:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f10d24d00) 00010000:00010000:3.0:1789302793.723312:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302793.723371:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:29391:x1876219350393856:12345-192.168.203.11@tcp:103:diff.0 Request processed in 139us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.723379:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 50 00000100:00100000:2.0:1789302793.728868:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.729211:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350393984 00000100:00100000:3.0:1789302793.729226:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350393984 00000100:00100000:3.0:1789302793.729318:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1144 00000100:00100000:3.0:1789302793.729333:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22330850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:29391:x1876219350393984:12345-192.168.203.11@tcp:58:diff.0 00010000:00010000:3.0:1789302793.729476:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.729496:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.729543:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302793.729880:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].32, 136 off12544 00000100:00100000:3.0:1789302793.730005:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22330850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:29391:x1876219350393984:12345-192.168.203.11@tcp:58:diff.0 Request processed in 675us (1148us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.730015:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1144 00000100:00100000:2.0:1789302793.739017:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.740299:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350394112 00000100:00100000:3.0:1789302793.740315:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350394112 00000100:00100000:3.0:1789302793.740398:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1145 00000100:00100000:3.0:1789302793.740411:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:29391:x1876219350394112:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302793.740438:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302793.740465:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0820 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.740567:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.740582:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 106 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.740605:0:5844:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.740621:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 106 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.740636:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302793.740639:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302793.740644:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 106 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.740652:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302793.740734:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:29391:x1876219350394112:12345-192.168.203.11@tcp:101:diff.0 Request processed in 326us (1730us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.740742:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1145 00000040:00080000:3.0:1789302793.743289:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302793.743301:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302793.743355:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 4 00000040:00080000:3.0:1789302793.743364:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302793.743582:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d9180 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371136768:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302793.743712:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302793.749980:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302793.750082:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d9180 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371136768:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302793.750594:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350394240 00000100:00100000:0.0:1789302793.750605:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350394240 00000100:00100000:0.0:1789302793.750686:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1146 00000100:00100000:0.0:1789302793.750698:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29392:x1876219350394240:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302793.750740:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302793.751712:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29392:x1876219350394240:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 1011us (1741us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302793.751722:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1146 00000100:00080000:2.0:1789302793.755395:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302793.758017:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.758209:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350394368 00000100:00100000:0.0:1789302793.758222:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350394368 00000100:00100000:0.0:1789302793.758302:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1147 00000100:00100000:0.0:1789302793.758315:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29392:x1876219350394368:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1789302793.758411:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758432:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758447:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758464:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758472:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758479:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758525:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758534:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758542:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302793.758593:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:0.0:1789302793.758598:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302793.758613:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758625:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758632:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758640:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758647:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302793.758651:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758660:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758674:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302793.758677:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302793.758682:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758689:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e4df lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758707:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758714:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758720:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758725:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ea800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302793.758728:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758734:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758742:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x329c2aeb (ffff960f03501600) 00010000:00010000:0.0:1789302793.758744:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302793.758747:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758766:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ea800/0x64fe43095184e4e6 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758781:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758788:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758794:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758800:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011eaa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302793.758803:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758809:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758816:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302793.758819:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302793.758822:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.758828:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e4ed lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302793.758911:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29392:x1876219350394368:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 598us (909us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302793.758920:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1147 00000100:00100000:2.0:1789302793.766424:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.768178:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350394624 00000100:00100000:0.0:1789302793.768197:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350394624 00000100:00100000:0.0:1789302793.768277:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1148 00000100:00100000:0.0:1789302793.768290:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29392:x1876219350394624:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302793.768319:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302793.768348:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e4f4 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0827 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768482:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768524:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768538:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302793.768550:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1789302793.768673:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768681:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768688:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768695:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302793.768699:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768708:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768724:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302793.768727:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302793.768733:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.768739:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e4fb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.769085:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e4f4 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0827 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.769106:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302793.769110:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302793.769114:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e4f4 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0827 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.769125:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01662800, rc 0) 00000100:00100000:0.0:1789302793.769253:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29392:x1876219350394624:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 965us (2843us total) trans 4294968052 rc 301/301 00000100:00100000:0.0:1789302793.769262:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1148 00000100:00100000:1.0:1789302793.781444:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.781872:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350394752 00000100:00100000:0.0:1789302793.781885:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350394752 00000100:00100000:0.0:1789302793.781996:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1149 00000100:00100000:0.0:1789302793.782009:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29392:x1876219350394752:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302793.782057:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302793.782218:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29392:x1876219350394752:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 211us (788us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302793.782227:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1149 00000100:00100000:1.0:1789302793.791293:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302793.793336:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350395008 00000100:00100000:2.0:1789302793.793356:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350395008 00000100:00100000:2.0:1789302793.793442:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1150 00000100:00100000:2.0:1789302793.793455:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29392:x1876219350395008:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302793.793496:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302793.793542:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e509 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a082e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302793.794439:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e510 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302793.794459:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e510 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302793.794473:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e510 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302793.794484:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x21:0x0]: returning size 0 00010000:00010000:2.0:1789302793.794572:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e509 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a082e expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302793.794598:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e510 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a082e expref: 47 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302793.794609:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f02d72800) 00010000:00010000:2.0:1789302793.794613:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302793.794617:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3800, rc 0) 00000100:00100000:2.0:1789302793.794703:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29392:x1876219350395008:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1250us (8494us total) trans 4294968053 rc 0/0 00000100:00100000:2.0:1789302793.794712:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1150 00000100:00100000:1.0:1789302793.802328:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.802423:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350395264 00000100:00100000:0.0:1789302793.802442:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350395264 00000100:00100000:0.0:1789302793.802553:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1151 00000100:00100000:0.0:1789302793.802569:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29392:x1876219350395264:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302793.802646:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302793.802683:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e510 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a082e expref: 47 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.802703:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f02d72800) 00010000:00010000:0.0:1789302793.802717:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302793.802747:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.802777:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 105 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.802787:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 4/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.802839:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 105 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.802856:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 105 pid: 5843 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302793.802873:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371136896:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302793.802995:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6240 going to sleep for 11 seconds 00000100:00100000:1.0:1789302793.806741:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.807936:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350395392 00000100:00100000:3.0:1789302793.807953:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350395392 00010000:00010000:3.0:1789302793.808004:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 106 pid: 5843 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302793.808056:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:3.0:1789302793.808067:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2348:x1876219350395392:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302793.808088:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302793.808104:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 106 pid: 5843 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.808120:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 106 pid: 5843 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.808146:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:3.0:1789302793.808152:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.808165:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.808173:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.808198:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302793.808912:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e429 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a07e1 expref: 106 pid: 5843 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302793.808932:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371136896:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302793.808959:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.808976:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.808983:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302793.809137:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:2348:x1876219350395392:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1070us (2404us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.809149:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 345 00010000:00010000:0.0:1789302793.809160:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809182:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809194:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809468:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809477:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809485:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809491:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01662800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302793.809496:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809545:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809558:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302793.809563:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302793.809569:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809575:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e51e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809592:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809599:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809605:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809610:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302793.809614:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809620:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809633:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x71322268 (ffff960f03501600) 00010000:00010000:0.0:1789302793.809636:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302793.809639:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.809645:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e525 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x71322268 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302793.810208:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29392:x1876219350395264:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 7638us (7883us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302793.810220:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1151 00000100:00100000:3.0:1789302793.817557:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.817606:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350395520 00000100:00100000:3.0:1789302793.817619:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350395520 00000100:00100000:3.0:1789302793.817695:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1152 00000100:00100000:3.0:1789302793.817709:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29392:x1876219350395520:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302793.817738:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302793.817778:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e52c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0835 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.823067:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.823112:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.823131:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.823547:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e52c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.823571:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.823580:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302793.823584:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302793.823588:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:3.0:1789302793.823680:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038db480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29392:x1876219350395520:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 5972us (6136us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.823691:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1152 00000100:00100000:3.0:1789302793.826554:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.827296:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350395776 00000100:00100000:0.0:1789302793.827309:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350395776 00000100:00100000:0.0:1789302793.827378:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1153 00000100:00100000:0.0:1789302793.827390:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29392:x1876219350395776:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302793.827522:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.827543:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.827559:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302793.828876:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ec71f26f reply state 00000000571cb778 transno 4294968054 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302793.828893:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302793.829109:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038dbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29392:x1876219350395776:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1708us (2554us total) trans 4294968054 rc 0/0 00000100:00100000:0.0:1789302793.829119:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1153 00010000:00010000:3.0:1789302793.832161:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.832194:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.832202:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.832208:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302793.832214:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.832221:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.832241:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302793.832243:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789302793.832249:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.832256:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e53a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302793.839835:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.840288:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350395904 00000100:00100000:0.0:1789302793.840303:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350395904 00000100:00100000:0.0:1789302793.840374:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1154 00000100:00100000:0.0:1789302793.840387:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29392:x1876219350395904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302793.840423:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302793.840453:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a083c expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.840566:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 49 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.840585:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 49 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.840597:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302793.840601:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302793.840606:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01663000, rc 0) 00000100:00100000:0.0:1789302793.840684:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29392:x1876219350395904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 299us (863us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302793.840692:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1154 00000100:00100000:2.0:1789302793.845205:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.845365:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350396032 00000100:00100000:3.0:1789302793.845380:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350396032 00000100:00100000:3.0:1789302793.845456:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 227 00000100:00100000:3.0:1789302793.845469:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29392:x1876219350396032:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1789302793.845673:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29392:x1876219350396032:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 205us (482us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.845681:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 227 00000100:00100000:3.0:1789302793.848894:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302793.849113:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350396288 00000100:00100000:2.0:1789302793.849141:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350396288 00000100:00100000:2.0:1789302793.849216:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 228 00000100:00100000:2.0:1789302793.849229:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ee050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29392:x1876219350396288:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302793.849579:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ee050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29392:x1876219350396288:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 352us (699us total) trans 4294968055 rc 0/0 00000100:00100000:2.0:1789302793.849588:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 228 00000100:00100000:2.0:1789302793.852212:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.852378:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350396544 00000100:00100000:3.0:1789302793.852391:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350396544 00000100:00100000:3.0:1789302793.852449:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1155 00000100:00100000:3.0:1789302793.852462:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29392:x1876219350396544:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302793.852636:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29392:x1876219350396544:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 176us (437us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.852644:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1155 00000100:00100000:2.0:1789302793.855852:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.855958:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350396800 00000100:00100000:0.0:1789302793.855972:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350396800 00000100:00100000:0.0:1789302793.859198:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1156 00000100:00100000:0.0:1789302793.859219:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29392:x1876219350396800:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302793.859251:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302793.859292:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e548 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0843 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.859439:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.859458:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.859474:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 4/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.859538:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.859554:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 51 pid: 5845 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302793.859570:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371137024:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302793.859688:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302793.864821:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302793.871072:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350396928 00000100:00100000:3.0:1789302793.871104:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350396928 00010000:00010000:3.0:1789302793.871163:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 52 pid: 5845 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302793.871220:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 346 00000100:00100000:3.0:1789302793.871233:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350396928:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302793.871251:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302793.871259:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 52 pid: 5845 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.871275:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 52 pid: 5845 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.871301:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302793.871308:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.871319:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.871329:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.871353:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302793.871431:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350396928:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 200us (6624us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.871440:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 346 00010000:00010000:1.0:1789302793.873692:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184e541 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a083c expref: 51 pid: 5845 timeout: 730 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302793.873719:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371137024:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302793.873817:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.873829:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.873853:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302793.873880:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x21:0x0]: returning size 0 00010000:00010000:1.0:1789302793.875497:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789302793.875732:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x18:0x0], index 0: 0 00010000:00010000:0.0:1789302793.875982:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876056:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876067:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876073:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302793.876078:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876087:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876117:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302793.876121:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302793.876127:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 2/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876133:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184e54f lrc: 0/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876262:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e548 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0843 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876275:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302793.876278:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302793.876284:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184e548 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0843 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.876295:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01662800, rc 0) 00000100:00100000:0.0:1789302793.876422:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29392:x1876219350396800:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 17209us (20581us total) trans 4294968056 rc 301/301 00000100:00100000:0.0:1789302793.876433:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1156 00000100:00100000:0.0:1789302793.881463:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302793.881633:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350397184 00000100:00100000:1.0:1789302793.881647:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350397184 00000100:00100000:2.0:1789302793.881866:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00100000:2.0:1789302793.881880:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29392:x1876219350397184:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302793.882357:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:29392:x1876219350397184:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 478us (904us total) trans 4294968057 rc 0/0 00000100:00100000:2.0:1789302793.882366:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 229 00000100:00100000:3.0:1789302793.884744:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302793.884848:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350397312 00000100:00100000:1.0:1789302793.884861:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350397312 00000100:00100000:1.0:1789302793.884927:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1157 00000100:00100000:1.0:1789302793.884939:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:29392:x1876219350397312:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302793.884969:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302793.885006:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e55d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a084a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.885107:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e564 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.885124:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e564 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.885137:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e564 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302793.885192:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x21:0x0]: returning size 0 00000004:00200000:1.0:1789302793.885198:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302793.885221:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e564 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.885252:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e55d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a084a expref: 53 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.885293:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e564 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a084a expref: 52 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.885303:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:1.0:1789302793.885306:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1789302793.885310:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6e00, rc 0) 00000100:00100000:1.0:1789302793.885388:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038db800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29392:x1876219350397312:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 451us (656us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302793.885396:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1157 00000100:00100000:3.0:1789302793.887817:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302793.888057:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350397440 00000100:00100000:1.0:1789302793.888070:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350397440 00000100:00100000:1.0:1789302793.888159:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00100000:1.0:1789302793.888171:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350397440:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302793.888257:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302793.888286:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0851 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302793.888320:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789302793.888346:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=630s v=(0 0) 00010000:00010000:1.0:1789302793.888351:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302793.888362:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.888381:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 10 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302793.888419:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7c00, rc 0) 00000100:00100000:1.0:1789302793.888487:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350397440:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 318us (682us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302793.888494:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00080000:1.0:1789302793.888538:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302793.892832:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.893391:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350397696 00000100:00100000:0.0:1789302793.893406:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350397696 00000100:00100000:0.0:1789302793.893474:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1158 00000100:00100000:0.0:1789302793.893487:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29392:x1876219350397696:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302793.894124:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29392:x1876219350397696:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 637us (1305us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302793.894136:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1158 00000100:00100000:2.0:1789302793.897032:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.897108:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350397824 00000100:00100000:0.0:1789302793.897120:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350397824 00000100:00100000:0.0:1789302793.897187:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 312 00000100:00100000:0.0:1789302793.897199:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:29392:x1876219350397824:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302793.897222:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302793.897247:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0858 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.897286:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.897296:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302793.897301:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302793.897307:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.897352:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.897368:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 11 pid: 6695 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302793.897384:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371137152:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302793.897748:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302793.903712:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.907859:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371137152:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302793.907913:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.907941:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011ea800, rc 0) 00000100:00100000:0.0:1789302793.908149:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:29392:x1876219350397824:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 10951us (11128us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302793.908161:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 312 00000100:00100000:3.0:1789302793.908497:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350397952 00000100:00100000:3.0:1789302793.908533:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350397952 00010000:00010000:3.0:1789302793.908573:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 11 pid: 6695 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302793.908612:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:3.0:1789302793.908622:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09751c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350397952:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302793.908639:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302793.908667:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 11 pid: 6695 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.908683:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 11 pid: 6695 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.908710:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e56b lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0851 expref: 11 pid: 6695 timeout: 730 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.908733:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302793.908738:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302793.908757:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.908770:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.908813:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302793.908830:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1876219371137280:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302793.908897:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0b40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302793.914976:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0000_UUID:5831:1876219371137280:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302793.915017:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302793.915093:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09751c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350397952:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 6470us (11392us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.915493:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:1.0:1789302793.915592:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.915666:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350398080 00000100:00100000:0.0:1789302793.915678:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350398080 00002000:00100000:0.0:1789302793.915737:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x18:0x0] (0->157) req@ffff960e01148a80 x1876219350398080/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:369/0 lens 648/0 e 0 to 0 dl 1789302804 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302793.915749:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e01148a80 with x1876219350398080 ext(0->157) 00002000:00010000:0.0:1789302793.915756:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e01148a80 00000100:00100000:0.0:1789302793.915782:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 104 00000100:00100000:0.0:1789302793.915793:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350398080:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1789302793.916553:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350398080:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 758us (958us total) trans 4294967361 rc 0/0 00000100:00100000:0.0:1789302793.916564:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 104 00002000:00100000:0.0:1789302793.916572:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x18:0x0] (0->157) req@ffff960e01148a80 x1876219350398080/t4294967361(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:369/0 lens 648/448 e 0 to 0 dl 1789302804 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302793.916581:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e01148a80 with x1876219350398080 ext(0->157) 00002000:00010000:0.0:1789302793.916586:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960e01148a80 00000100:00100000:2.0:1789302793.919296:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.919436:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350398208 00000100:00100000:0.0:1789302793.919450:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350398208 00000100:00100000:0.0:1789302793.919542:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1159 00000100:00100000:0.0:1789302793.919555:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29392:x1876219350398208:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1789302793.966803:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968054 is committed 00000001:00080000:3.0:1789302793.966828:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968042 is committed 00000001:00080000:3.0:1789302793.966854:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968056 is committed 00000001:00080000:3.0:1789302793.966873:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968057 is committed 00000001:00080000:3.0:1789302793.966886:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968050 is committed 00000004:00080000:3.0:1789302793.966891:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 11 committed 00000020:00080000:3.0:1789302793.966912:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f3ff9ba80 refcount 2 committed 0 result 0 batchid 23 00000020:00080000:3.0:1789302793.966919:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09762700 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f3ff9b3c0 00000020:00080000:3.0:1789302793.966923:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09762000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f25c59800 00000020:00080000:3.0:1789302793.967106:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 23 committed batchid 22 00000020:00080000:3.0:1789302793.967685:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 22 00000100:00100000:3.0:1789302793.967710:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371137408:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789302793.967805:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302793.968027:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29392:x1876219350398208:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 48472us (48742us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302793.968038:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1159 00000001:00080000:2.0:1789302793.969008:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968051 is committed 00000100:00100000:0.0:1789302793.971885:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371137408 00000100:00100000:0.0:1789302793.971903:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371137408 00000100:00100000:0.0:1789302793.971999:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 313 00000100:00100000:0.0:1789302793.972022:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371137408:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1789302793.972061:0:18578:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:23 count 1 00002000:00010000:0.0:1789302793.972116:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x17:0x0] initial LVB size: 158, mtime: 0x6aa69804, atime: 0x0, ctime: 0x6aa69804, blocks: 0xb 00010000:00010000:0.0:1789302793.972126:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.972146:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=630s v=(0 0) 00010000:00010000:0.0:1789302793.972151:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x17:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302793.972158:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.972174:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.972184:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.972193:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302793.972211:0:18578:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302793.972896:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3303:x1876219371137408:12345-0@lo:6:osp-syn-1-0.0 Request processed in 877us (5088us total) trans 4294967362 rc 0/0 00000100:00100000:0.0:1789302793.972905:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 313 00000100:00080000:2.0:1789302793.973212:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1789302793.973626:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302793.973637:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.973649:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302793.973654:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.973666:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.973691:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.973699:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000005f0fe3e/0x64fe43095184e579 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x17:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302793.973728:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302793.973730:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302793.973732:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789302793.973783:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e038daa00/2, rc 0, transno 4294967362 00000100:00100000:3.0:1789302793.973791:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371137408:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789302793.974330:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302793.977831:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350398336 00000100:00100000:0.0:1789302793.977849:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350398336 00000100:00100000:0.0:1789302793.977935:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 314 00000100:00100000:0.0:1789302793.977947:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350398336:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1789302794.014194:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967361 is committed 00000100:00100000:1.0:1789302794.014410:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350398336:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 36457us (40075us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302794.014424:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 314 00000100:00100000:3.0:1789302794.016969:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302794.017065:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350398592 00000100:00100000:1.0:1789302794.017078:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350398592 00000100:00100000:1.0:1789302794.017185:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1160 00000100:00100000:1.0:1789302794.017196:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29392:x1876219350398592:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302794.017343:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302794.017381:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e564 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a084a expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.017404:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:1.0:1789302794.017409:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302794.017445:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.017462:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.017493:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302794.017741:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000a502d3da reply state 0000000013b88e96 transno 4294968058 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302794.017762:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302794.017873:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29392:x1876219350398592:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 678us (915us total) trans 4294968058 rc 0/0 00000100:00100000:1.0:1789302794.017880:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1160 00000100:00100000:2.0:1789302794.020056:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302794.020558:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350398720 00000100:00100000:0.0:1789302794.020572:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350398720 00000100:00100000:0.0:1789302794.020643:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1161 00000100:00100000:0.0:1789302794.020655:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29392:x1876219350398720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302794.020687:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302794.020715:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a085f expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.020800:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.020817:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.020841:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302794.020845:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302794.020850:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e011eaa00, rc 0) 00000100:00100000:0.0:1789302794.020915:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29392:x1876219350398720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 263us (869us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302794.020923:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1161 00010000:00010000:3.0:1789302794.021059:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.021068:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.021075:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.021080:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302794.021085:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.021093:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.021116:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302794.021119:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302794.021123:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.021129:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e580 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302794.023004:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302794.023054:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350398848 00000100:00100000:3.0:1789302794.023065:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350398848 00000100:00100000:3.0:1789302794.023134:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:3.0:1789302794.023146:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350398848:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1789302794.024622:0:13439:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1789302794.024632:0:13439:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb mtime from disk: 0 -> 1789302788 00002000:00010000:3.0:1789302794.024635:0:13439:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb atime from disk: 0 -> 1789302788 00002000:00010000:3.0:1789302794.024637:0:13439:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb ctime from disk: 0 -> 1789302788 00002000:00010000:3.0:1789302794.024640:0:13439:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x18:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1789302794.024782:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350398848:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 1633us (1786us total) trans 4294967362 rc 0/0 00000100:00100000:3.0:1789302794.024791:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:0.0:1789302794.026971:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302794.027066:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350399104 00000100:00100000:3.0:1789302794.027078:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350399104 00000100:00100000:3.0:1789302794.027190:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1162 00000100:00100000:3.0:1789302794.027202:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29392:x1876219350399104:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1789302794.027281:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.027303:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.027314:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 4/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.027365:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.027379:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 47 pid: 18427 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302794.027396:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371137536:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302794.027527:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302794.030560:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302794.030594:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350399232 00000100:00100000:3.0:1789302794.030605:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350399232 00010000:00010000:3.0:1789302794.030651:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 48 pid: 18427 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302794.030689:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 348 00000100:00100000:3.0:1789302794.030699:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038db100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2347:x1876219350399232:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302794.030717:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302794.030725:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 48 pid: 18427 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.030740:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 48 pid: 18427 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.030778:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302794.030783:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.030797:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.030806:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.030827:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302794.030876:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038db100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2347:x1876219350399232:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 179us (329us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302794.030883:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 348 00010000:00010000:1.0:1789302794.031906:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184e533 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x21:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0835 expref: 47 pid: 18427 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302794.031928:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371137536:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302794.031943:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.031953:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.031959:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.031997:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.032005:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.032034:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.032041:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302794.032046:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.032055:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.032068:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:1.0:1789302794.032071:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1789302794.032075:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.032080:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e58e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302794.033450:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29392:x1876219350399104:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 6238us (6477us total) trans 0 rc -22/-22 00000100:00100000:1.0:1789302794.033466:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1162 00000100:00100000:0.0:1789302794.035491:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302794.035742:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350399360 00000100:00100000:2.0:1789302794.035820:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350399360 00000100:00100000:2.0:1789302794.035922:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 316 00000100:00100000:2.0:1789302794.035935:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350399360:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302794.035996:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.036037:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.036049:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x18:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789302794.036054:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302794.036059:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.036122:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.036138:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 11 pid: 6697 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302794.036156:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371137664:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302794.036302:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302794.039245:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302794.039367:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350399488 00000100:00100000:3.0:1789302794.039379:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350399488 00010000:00010000:3.0:1789302794.039431:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 12 pid: 6697 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302794.039480:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 349 00000100:00100000:3.0:1789302794.039492:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350399488:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302794.039543:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302794.039573:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 12 pid: 6697 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.039591:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 12 pid: 6697 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.039624:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302794.039629:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302794.039636:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.039649:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.039675:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302794.039745:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350399488:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 254us (514us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302794.039761:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 349 00010000:00010000:0.0:1789302794.041325:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000067e04838/0x64fe43095184e572 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0858 expref: 11 pid: 6697 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302794.041349:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371137664:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302794.041377:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041389:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041396:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041433:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041442:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041452:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041461:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302794.041466:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041477:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041499:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.041619:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e595 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302794.041766:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350399360:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 5833us (6287us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302794.041776:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 316 00000100:00100000:2.0:1789302794.047444:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302794.047668:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350399744 00000100:00100000:0.0:1789302794.047682:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350399744 00000100:00100000:0.0:1789302794.047850:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1163 00000100:00100000:0.0:1789302794.047863:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29392:x1876219350399744:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302794.047969:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.047991:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.048006:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.048341:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.048355:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.048364:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.048404:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302794.048419:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 47 pid: 6919 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302794.048435:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371137792:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302794.048759:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f123a6780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302794.052695:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef6300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371137792:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302794.052737:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302794.052878:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302794.053008:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350399872 00000100:00100000:3.0:1789302794.053018:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350399872 00010000:00010000:3.0:1789302794.053064:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 48 pid: 6919 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302794.053143:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 350 00000100:00100000:3.0:1789302794.053154:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350399872:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302794.053181:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302794.053188:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 48 pid: 6919 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.053202:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 48 pid: 6919 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.053228:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011eaa00/0x64fe43095184e587 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a085f expref: 48 pid: 6919 timeout: 731 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.053245:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302794.053251:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.053269:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.053277:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302794.053342:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302794.053398:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350399872:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 245us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302794.053406:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 350 00010000:00010000:2.0:1789302794.054445:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.054459:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302794.054896:0:6919:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968059 00000004:02000000:2.0:1789302794.055071:0:6919:0:(mdd_object.c:3045:mdd_swap_layouts()) *** cfs_fail_loc=152, val=0*** 00000001:00080000:2.0:1789302794.060277:0:6919:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968060 00010000:00010000:2.0:1789302794.060368:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060382:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060390:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060397:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302794.060403:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060412:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060432:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:2.0:1789302794.060436:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302794.060441:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060448:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5a3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x21:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060491:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060499:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060537:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060561:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302794.060565:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060572:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060582:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:2.0:1789302794.060584:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302794.060594:0:6919:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060606:0:6919:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060616:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302794.060667:0:6919:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302794.060687:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb9880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371137920:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302794.060962:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302794.064950:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302794.065076:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350400000 00000100:00100000:3.0:1789302794.065089:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350400000 00000100:00100000:3.0:1789302794.065181:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 230 00000100:00100000:3.0:1789302794.065202:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29391:x1876219350400000:12345-192.168.203.11@tcp:49:diff.0 00000100:00100000:3.0:1789302794.065380:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29391:x1876219350400000:12345-192.168.203.11@tcp:49:diff.0 Request processed in 180us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302794.065388:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 230 00000100:00100000:1.0:1789302794.065607:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb9880 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371137920:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302794.065640:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065667:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e4d1 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065707:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065724:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065737:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065749:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065757:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065764:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065770:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302794.065775:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065783:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065796:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302794.065800:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302794.065804:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065817:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e5aa lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065829:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065835:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065841:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065846:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e011ebc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302794.065850:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065856:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065864:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302794.065866:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302794.065870:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.065875:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e011ebc00/0x64fe43095184e59c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302794.066140:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29392:x1876219350399744:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 18279us (18708us total) trans 0 rc -95/-95 00000100:00100000:1.0:1789302794.066149:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1163 00000020:00080000:1.0:1789302794.066159:0:6919:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000040:00080000:3.0:1789302794.066560:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302794.066566:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302794.067052:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1789302794.067062:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302794.078789:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302794.078825:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302794.078908:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350400256 00000100:00100000:1.0:1789302794.078922:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350400256 00000100:00100000:1.0:1789302794.078993:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350400512 00000100:00100000:1.0:1789302794.078997:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350400512 00000100:00100000:1.0:1789302794.079021:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 231 00000100:00100000:1.0:1789302794.079032:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29391:x1876219350400256:12345-192.168.203.11@tcp:35:diff.0 00000100:00100000:1.0:1789302794.079365:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29391:x1876219350400256:12345-192.168.203.11@tcp:35:diff.0 Request processed in 333us (591us total) trans 4294968061 rc 0/0 00000100:00100000:1.0:1789302794.079374:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 231 00000100:00100000:1.0:1789302794.079394:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 232 00000100:00100000:1.0:1789302794.079401:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29392:x1876219350400512:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1789302794.080262:0:5847:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x21:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1789302794.080428:0:5847:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].16, 64 off9264 00000004:00080000:1.0:1789302794.080442:0:5847:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 631090938895ns, next at 632090936373ns, rc = 0 00000001:00080000:1.0:1789302794.080451:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968062 00010000:00010000:1.0:1789302794.080545:0:5847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080566:0:5847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080581:0:5847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080591:0:5847:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080598:0:5847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080614:0:5847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080634:0:5847:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1789302794.080719:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29392:x1876219350400512:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1315us (1889us total) trans 4294968063 rc 0/0 00000100:00100000:1.0:1789302794.080728:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 232 00000020:00080000:1.0:1789302794.080734:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1789302794.080766:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302794.080772:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080779:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302794.080784:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080793:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080807:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x21:0x0].0x0 (ffff960f421e9400) 00010000:00010000:1.0:1789302794.080810:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789302794.080816:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080823:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e5b1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x21:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302794.080840:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302794.080842:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302794.080844:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302794.785728:0:20687:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302794.785737:0:20687:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302794.785785:0:20687:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 4 00000040:00080000:1.0:1789302794.785793:0:20687:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302794.893595:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302795.917826:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302796.042863:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302796.042993:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350400768 00000100:00100000:0.0:1789302796.043003:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350400768 00000100:00100000:0.0:1789302796.043079:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1164 00000100:00100000:0.0:1789302796.043098:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29414:x1876219350400768:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302796.043122:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302796.043142:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e5b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0866 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043216:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043231:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043245:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043260:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043267:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043274:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043554:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5cd lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043564:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5cd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043571:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5cd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302796.043580:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1789302796.043656:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043664:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043670:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043675:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302796.043680:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043686:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043697:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302796.043713:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302796.043717:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043723:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5bf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043736:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043742:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043747:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043751:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302796.043754:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043760:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043767:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x28d475f8 (ffff960f03501800) 00010000:00010000:0.0:1789302796.043770:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302796.043773:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043778:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x46:0x0].0x28d475f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043820:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e5b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0866 expref: 99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043834:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5cd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0866 expref: 98 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302796.043843:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302796.043845:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302796.043848:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044400, rc 0) 00000100:00100000:0.0:1789302796.043912:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29414:x1876219350400768:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 815us (1056us total) trans 4294968064 rc 0/0 00000100:00100000:0.0:1789302796.043920:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1164 00000100:00080000:3.0:1789302796.044917:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302796.049640:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302796.049728:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350400896 00000100:00100000:3.0:1789302796.049740:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350400896 00000100:00100000:3.0:1789302796.049796:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1165 00000100:00100000:3.0:1789302796.049807:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29414:x1876219350400896:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:3.0:1789302796.049869:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049886:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049898:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049927:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049935:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049942:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049948:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302796.049953:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049960:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049972:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302796.049975:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302796.049980:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302796.049990:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e5db lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302796.050051:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29414:x1876219350400896:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 246us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.050058:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1165 00000100:00100000:0.0:1789302796.058326:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302796.059412:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350401152 00000100:00100000:3.0:1789302796.059422:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350401152 00000100:00100000:3.0:1789302796.059482:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 233 00000100:00100000:3.0:1789302796.059492:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29414:x1876219350401152:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:3.0:1789302796.059864:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:29414:x1876219350401152:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 372us (1545us total) trans 4294968065 rc 0/0 00000100:00100000:3.0:1789302796.059872:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 233 00000100:00100000:1.0:1789302796.303670:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371138048:0@lo:400:kworker.0 00000100:00100000:1.0:1789302796.303765:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302796.303939:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138048 00000100:00100000:3.0:1789302796.303949:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138048 00000100:00100000:3.0:1789302796.303995:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 572 00000100:00100000:3.0:1789302796.304005:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371138048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.314921:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371138048:12345-0@lo:400:kworker.0 Request processed in 10908us (11146us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.314935:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 572 00000100:00100000:3.0:1789302796.315114:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371138304:0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.315161:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302796.315210:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138304 00000100:00100000:3.0:1789302796.315217:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138304 00000100:00100000:3.0:1789302796.315262:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:3.0:1789302796.315270:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371138304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.315316:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371138304:12345-0@lo:400:kworker.0 Request processed in 46us (155us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.315322:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00080000:3.0:1789302796.315338:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302796.315380:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371138304:0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.315419:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371138432:0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.315446:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302796.315475:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138432 00000100:00100000:3.0:1789302796.315478:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138432 00000100:00100000:3.0:1789302796.315528:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:3.0:1789302796.315535:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371138432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.315570:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371138432:12345-0@lo:400:kworker.0 Request processed in 35us (123us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.315576:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00080000:3.0:1789302796.315589:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302796.315607:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371138432:0@lo:400:kworker.0 00000100:00100000:0.0:1789302796.317620:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371138176:0@lo:400:kworker.0 00000100:00100000:0.0:1789302796.317685:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302796.317809:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138176 00000100:00100000:3.0:1789302796.317817:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138176 00000100:00100000:3.0:1789302796.317859:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:3.0:1789302796.317868:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371138176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.317915:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371138176:12345-0@lo:400:kworker.0 Request processed in 48us (229us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.317921:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:0.0:1789302796.318033:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371138176:0@lo:400:kworker.0 00000100:00100000:0.0:1789302796.318042:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371138688:0@lo:400:kworker.0 00000100:00100000:0.0:1789302796.318068:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302796.318111:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302796.318118:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371138560:0@lo:400:kworker.0 00000100:00100000:0.0:1789302796.318139:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302796.318175:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371138048:0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.318223:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138688 00000100:00100000:3.0:1789302796.318228:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138688 00000100:00100000:3.0:1789302796.318269:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:3.0:1789302796.318276:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371138688:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302796.318312:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859bc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371138688:12345-0@lo:400:kworker.0 Request processed in 37us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.318319:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:3.0:1789302796.318490:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371138688:0@lo:400:kworker.0 00000100:00080000:0.0:1789302796.318622:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302796.319148:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138560 00000100:00100000:1.0:1789302796.319154:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138560 00000100:00100000:1.0:1789302796.319205:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1789302796.319213:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371138560:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302796.319271:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371138560:12345-0@lo:400:kworker.0 Request processed in 59us (1130us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302796.319278:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00080000:0.0:1789302796.319972:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302796.321159:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371138560:0@lo:400:kworker.0 00000001:00080000:2.0:1789302796.408485:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967362 is committed 00000100:00100000:3.0:1789302796.559212:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302796.559260:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302796.559289:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302796.559366:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302796.559399:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350401408 00000100:00100000:3.0:1789302796.559408:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350401408 00000100:00100000:3.0:1789302796.559463:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350401536 00000100:00100000:3.0:1789302796.559466:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350401536 00000100:00100000:3.0:1789302796.559483:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350401792 00000100:00100000:3.0:1789302796.559485:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350401792 00000100:00100000:3.0:1789302796.559530:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 420 00000100:00100000:3.0:1789302796.559541:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350401408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302796.559610:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038dad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350401408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 71us (408us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.559618:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 420 00000100:00100000:3.0:1789302796.559634:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 421 00000100:00100000:3.0:1789302796.559640:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350401536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302796.559666:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350401536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (376us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.559676:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 421 00000100:00100000:3.0:1789302796.559688:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 422 00000100:00100000:3.0:1789302796.559694:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350401792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302796.559729:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350401792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302796.559744:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 422 00000100:00080000:3.0:1789302796.559762:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302796.559766:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302796.560138:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350401280 00000100:00100000:1.0:1789302796.560144:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350401280 00000100:00100000:1.0:1789302796.560183:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:1.0:1789302796.560191:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350401280:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302796.560228:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350401280:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (967us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302796.560234:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:3.0:1789302796.577939:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302796.583949:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302796.584030:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350401920 00000100:00100000:0.0:1789302796.584041:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350401920 00000100:00100000:0.0:1789302796.584129:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 423 00000100:00100000:0.0:1789302796.584140:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350401920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302796.584329:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038da680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350401920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 190us (388us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302796.584338:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 423 00000100:00080000:3.0:1789302796.584532:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302796.585694:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350401664 00000100:00100000:1.0:1789302796.585704:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350401664 00000100:00100000:1.0:1789302796.585772:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 423 00000100:00100000:1.0:1789302796.585781:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2347:x1876219350401664:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302796.587044:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2347:x1876219350401664:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1258us (9109us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302796.587069:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 423 00000100:00080000:3.0:1789302796.589002:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302796.820306:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302796.820329:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371138816:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302796.820393:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302796.822408:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138816 00000100:00100000:1.0:1789302796.822423:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138816 00000100:00100000:1.0:1789302796.822586:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1789302796.822598:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371138816:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302796.822638:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302796.822716:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371138816:12345-0@lo:13:osp-pre-1-0.0 Request processed in 120us (2322us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302796.822723:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:1.0:1789302796.828127:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967362 gen 2 00000004:00080000:1.0:1789302796.828139:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967361 committed transno 4294967362 00000004:00080000:1.0:1789302796.828336:0:3303:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e038daa00, transno 4294967362 00000100:00100000:1.0:1789302796.828399:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371138816:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302796.943683:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789302797.326295:0:6999:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 634336788117ns, next at 635336784535ns, rc = 0 00000100:00100000:2.0:1789302797.326385:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302797.326426:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371138944:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302797.326537:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302797.327032:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371138944 00000100:00100000:3.0:1789302797.327044:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371138944 00000100:00100000:3.0:1789302797.327131:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 183 00000100:00100000:3.0:1789302797.327151:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371138944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302797.327201:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302797.327276:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371138944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 135us (739us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302797.327283:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 183 00000100:00100000:2.0:1789302797.327456:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371138944:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1789302797.967285:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302798.988880:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302799.181000:0:20833:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302799.181014:0:20833:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302799.181055:0:20833:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 4 00000040:00080000:2.0:1789302799.181073:0:20833:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:2.0:1789302799.449276:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967362 is committed 00000001:00080000:2.0:1789302799.499882:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968064 is committed 00000001:00080000:2.0:1789302799.499906:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968059 is committed 00000001:00080000:2.0:1789302799.499916:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968063 is committed 00000004:00080000:2.0:1789302799.499923:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 17 committed 00000004:00080000:2.0:1789302799.499967:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 18 committed 00000001:00080000:2.0:1789302799.499977:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968065 is committed 00000020:00080000:2.0:1789302799.499983:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 23 batchid updated 00000020:00080000:2.0:1789302799.500004:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 23 00000020:00080000:2.0:1789302799.500018:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 23 00000100:00100000:1.0:1789302799.501713:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371139072:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789302799.501780:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302799.501855:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139072 00000100:00100000:1.0:1789302799.501862:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139072 00000100:00100000:1.0:1789302799.501910:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 317 00000100:00100000:1.0:1789302799.501917:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371139072:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1789302799.501952:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:24 count 1 00002000:00010000:1.0:1789302799.501983:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x18:0x0] initial LVB size: 158, mtime: 0x6aa69804, atime: 0x6aa69804, ctime: 0x6aa69804, blocks: 0xb 00010000:00010000:1.0:1789302799.501992:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302799.502006:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=636s v=(0 0) 00010000:00010000:1.0:1789302799.502022:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302799.502028:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302799.502043:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302799.502067:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302799.502076:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302799.502102:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302799.502815:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302799.502830:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302799.502844:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302799.502850:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302799.502862:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302799.502885:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302799.502894:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe43095184e5e2 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302799.502915:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302799.502918:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302799.502920:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302799.521010:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371139072:12345-0@lo:6:osp-syn-0-0.0 Request processed in 19085us (19221us total) trans 4294967363 rc 0/0 00000100:00100000:1.0:1789302799.521024:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 317 00000100:00080000:1.0:1789302799.521084:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000004:00080000:3.0:1789302799.521886:0:3300:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e01148e00/2, rc 0, transno 4294967363 00000100:00100000:3.0:1789302799.521903:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371139072:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1789302800.012999:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302800.207893:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302800.207905:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302800.208391:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 33 count 2 00000040:00080000:2.0:1789302800.208400:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302801.036689:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302801.429664:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371139200:0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.429898:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302801.429957:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371139712:0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.429988:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302801.430018:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302801.430024:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371139584:0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.430064:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302801.430644:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139200 00000100:00100000:0.0:1789302801.430655:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139200 00000100:00100000:1.0:1789302801.431535:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139712 00000100:00100000:1.0:1789302801.431605:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139712 00000100:00100000:1.0:1789302801.431670:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:1.0:1789302801.431681:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371139712:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.431796:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:0.0:1789302801.431806:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371139200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.431897:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371139200:12345-0@lo:400:kworker.0 Request processed in 92us (1998us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302801.431904:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00000100:00100000:0.0:1789302801.431952:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371139456:0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.431985:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302801.432017:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789302801.432024:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371139328:0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.432058:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302801.432076:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371139840:0@lo:400:kworker.0 00000100:00100000:3.0:1789302801.439908:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139456 00000100:00100000:3.0:1789302801.439938:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139456 00000100:00100000:3.0:1789302801.440023:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139328 00000100:00100000:3.0:1789302801.440027:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139328 00000100:00100000:3.0:1789302801.440059:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:3.0:1789302801.440072:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371139456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.441314:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:1.0:1789302801.441331:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371139328:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.445240:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302801.445359:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371139328:0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.445447:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139584 00000100:00100000:0.0:1789302801.445456:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139584 00000100:00100000:0.0:1789302801.445535:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139840 00000100:00100000:0.0:1789302801.445539:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139840 00000100:00100000:0.0:1789302801.445583:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1789302801.445593:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371139584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.445677:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371139584:12345-0@lo:400:kworker.0 Request processed in 85us (15610us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302801.445684:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1789302801.445716:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:0.0:1789302801.445723:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371139840:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302801.445758:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371139840:12345-0@lo:400:kworker.0 Request processed in 35us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302801.445764:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:0.0:1789302801.445801:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371139840:0@lo:400:kworker.0 00000100:00080000:2.0:1789302801.447637:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302801.447646:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302801.447649:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302801.450982:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371139456:12345-0@lo:400:kworker.0 Request processed in 10909us (18993us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302801.450999:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:2.0:1789302801.451077:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371139456:0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.451271:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371139328:12345-0@lo:400:kworker.0 Request processed in 9944us (19208us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302801.451280:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:1.0:1789302801.451328:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371139584:0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.451344:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01148380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371139200:0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.451410:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371139712:12345-0@lo:400:kworker.0 Request processed in 19731us (21420us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302801.451418:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:2.0:1789302801.451498:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371139712:0@lo:400:kworker.0 00000100:00100000:1.0:1789302801.678600:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302801.678646:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302801.678671:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302801.678700:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302801.678723:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302801.678743:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302801.678767:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302801.678801:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402432 00000100:00100000:1.0:1789302801.678814:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402432 00000100:00100000:1.0:1789302801.678883:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402688 00000100:00100000:1.0:1789302801.678887:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402688 00000100:00100000:1.0:1789302801.678907:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402304 00000100:00100000:1.0:1789302801.678911:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402304 00000100:00100000:1.0:1789302801.678924:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402816 00000100:00100000:1.0:1789302801.678927:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402816 00000100:00100000:1.0:1789302801.678967:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 426 00000100:00100000:1.0:1789302801.678979:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350402432:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:2.0:1789302801.679039:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302801.679061:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302801.679079:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 427 00000100:00100000:2.0:1789302801.679088:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350402688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302801.679176:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350402688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 90us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302801.679184:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 427 00000100:00100000:2.0:1789302801.679206:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 428 00000100:00100000:2.0:1789302801.679212:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350402304:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302801.679238:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350402304:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302801.679244:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 428 00000100:00100000:2.0:1789302801.679255:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 429 00000100:00100000:2.0:1789302801.679261:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350402816:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302801.679297:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350402816:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (519us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302801.679303:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 429 00000100:00100000:3.0:1789302801.680066:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402176 00000100:00100000:3.0:1789302801.680076:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402176 00000100:00100000:3.0:1789302801.680163:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402560 00000100:00100000:3.0:1789302801.680167:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402560 00000100:00080000:2.0:1789302801.680675:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302801.681092:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 427 00000100:00100000:3.0:1789302801.681103:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350402176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302801.681179:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350402176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 79us (2595us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302801.681187:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 427 00000100:00100000:3.0:1789302801.681207:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 428 00000100:00100000:3.0:1789302801.681213:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350402560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302801.681242:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350402560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (2498us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302801.681249:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 428 00000100:00100000:1.0:1789302801.681748:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402048 00000100:00100000:1.0:1789302801.681758:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402048 00000100:00100000:1.0:1789302801.686711:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 575 00000100:00100000:1.0:1789302801.686742:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350402048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302801.686841:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350402048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 114us (8139us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302801.686850:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 575 00000100:00100000:1.0:1789302801.692755:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350402432:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 13760us (14090us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302801.692775:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 426 00000100:00100000:3.0:1789302801.934905:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302801.934936:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371139968:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302801.935023:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302801.935122:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371139968 00000100:00100000:3.0:1789302801.935131:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371139968 00000100:00100000:3.0:1789302801.935192:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:3.0:1789302801.935203:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371139968:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302801.935249:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302801.935313:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371139968:12345-0@lo:13:osp-pre-1-0.0 Request processed in 102us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302801.935320:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:3.0:1789302801.935594:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371139968:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302802.063687:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302802.445169:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371140096:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302802.445270:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302802.445996:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140096 00000100:00100000:3.0:1789302802.446008:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140096 00000100:00100000:3.0:1789302802.446093:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 185 00000100:00100000:3.0:1789302802.446105:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371140096:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302802.446152:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302802.446232:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371140096:12345-0@lo:13:osp-pre-0-0.0 Request processed in 129us (961us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302802.446239:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 185 00000100:00100000:2.0:1789302802.446361:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371140096:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302803.084850:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302804.110438:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1789302804.588824:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967363 is committed 00010000:00010000:3.0:1789302805.133312:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302806.156907:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302806.541466:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371140352:0@lo:400:kworker.0 00000100:00100000:3.0:1789302806.541836:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302806.541908:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371140864:0@lo:400:kworker.0 00000100:00100000:3.0:1789302806.541941:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302806.541973:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302806.541980:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371140480:0@lo:400:kworker.0 00000100:00100000:3.0:1789302806.542005:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302806.542053:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140352 00000100:00100000:3.0:1789302806.542064:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140352 00000100:00100000:3.0:1789302806.542114:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140480 00000100:00100000:3.0:1789302806.542118:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140480 00000100:00100000:2.0:1789302806.542346:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371140224:0@lo:400:kworker.0 00000100:00100000:2.0:1789302806.542445:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302806.542474:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371140736:0@lo:400:kworker.0 00000100:00100000:0.0:1789302806.542494:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140864 00000100:00100000:2.0:1789302806.542498:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302806.542526:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140864 00000100:00100000:0.0:1789302806.544733:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:0.0:1789302806.544756:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371140864:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302806.544876:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371140864:12345-0@lo:400:kworker.0 Request processed in 130us (2933us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302806.544884:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:0.0:1789302806.544930:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371140608:0@lo:400:kworker.0 00000100:00100000:0.0:1789302806.544970:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302806.547308:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140608 00000100:00100000:0.0:1789302806.547399:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140608 00000100:00100000:1.0:1789302806.551529:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140736 00000100:00100000:1.0:1789302806.551547:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140736 00000100:00100000:1.0:1789302806.552222:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:1.0:1789302806.552238:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371140352:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302806.552346:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371140352:12345-0@lo:400:kworker.0 Request processed in 113us (10510us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302806.552354:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00000100:00100000:1.0:1789302806.552375:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:1.0:1789302806.552383:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371140736:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302806.552421:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371140736:12345-0@lo:400:kworker.0 Request processed in 40us (9921us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302806.552448:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:0.0:1789302806.557915:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789302806.557949:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371140608:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302806.558092:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371140608:12345-0@lo:400:kworker.0 Request processed in 162us (13120us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302806.558099:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789302806.558170:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967363 gen 2 00000004:00080000:0.0:1789302806.558177:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e01148e00, transno 4294967363 00000100:00100000:0.0:1789302806.558203:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371140608:0@lo:400:kworker.0 00000100:00080000:0.0:1789302806.558875:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302806.558883:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302806.558886:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302806.558972:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140224 00000100:00100000:0.0:1789302806.558982:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140224 00000100:00100000:0.0:1789302806.559030:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 576 00000100:00100000:3.0:1789302806.559035:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789302806.559038:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371140224:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302806.559044:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371140480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302806.559265:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371140736:0@lo:400:kworker.0 00000100:00100000:2.0:1789302806.559451:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371140352:0@lo:400:kworker.0 00000100:00100000:2.0:1789302806.559464:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fba680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371140864:0@lo:400:kworker.0 00000100:00100000:0.0:1789302806.560224:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371140224:12345-0@lo:400:kworker.0 Request processed in 1184us (17775us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302806.560236:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 576 00000100:00100000:0.0:1789302806.560289:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371140224:0@lo:400:kworker.0 00000100:00100000:3.0:1789302806.561860:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371140480:12345-0@lo:400:kworker.0 Request processed in 2809us (19845us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302806.561874:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:0.0:1789302806.561896:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371140480:0@lo:400:kworker.0 00000100:00100000:1.0:1789302806.797604:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302806.797645:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302806.797675:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302806.797692:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302806.797716:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302806.797734:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302806.797758:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302806.797787:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350403200 00000100:00100000:1.0:1789302806.797797:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350403200 00000100:00100000:1.0:1789302806.797852:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350403712 00000100:00100000:1.0:1789302806.797855:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350403712 00000100:00100000:1.0:1789302806.797875:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350403328 00000100:00100000:1.0:1789302806.797878:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350403328 00000100:00100000:1.0:1789302806.797888:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350403584 00000100:00100000:1.0:1789302806.797890:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350403584 00000100:00100000:1.0:1789302806.797904:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 432 00000100:00100000:1.0:1789302806.797913:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350403200:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302806.797976:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350403200:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 64us (330us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302806.797984:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 432 00000100:00100000:1.0:1789302806.798000:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00100000:1.0:1789302806.798018:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350403712:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302806.798044:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350403712:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302806.798050:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00100000:1.0:1789302806.798079:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 434 00000100:00100000:1.0:1789302806.798086:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350403328:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302806.798124:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350403328:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (388us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302806.798130:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 434 00000100:00100000:1.0:1789302806.798142:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 435 00000100:00100000:1.0:1789302806.798150:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350403584:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302806.798175:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350403584:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302806.798182:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 435 00000100:00100000:0.0:1789302806.798221:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350403456 00000100:00100000:0.0:1789302806.798235:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350403456 00000100:00080000:1.0:1789302806.798249:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302806.798260:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302806.798262:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302806.798332:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350403072 00000100:00100000:0.0:1789302806.798337:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350403072 00000100:00100000:0.0:1789302806.798379:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 432 00000100:00100000:0.0:1789302806.798391:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350403456:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302806.798422:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350402944 00000100:00100000:3.0:1789302806.798430:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350402944 00000100:00100000:0.0:1789302806.798472:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350403456:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 78us (790us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302806.798473:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:0.0:1789302806.798494:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 432 00000100:00100000:3.0:1789302806.798495:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350402944:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302806.798561:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00100000:0.0:1789302806.798580:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350403072:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302806.798581:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350402944:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 86us (976us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302806.798603:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:0.0:1789302806.798632:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350403072:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 59us (911us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302806.798642:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00080000:3.0:1789302806.798655:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302807.053676:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371140992:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302807.053790:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302807.053876:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371140992 00000100:00100000:3.0:1789302807.053888:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371140992 00000100:00100000:3.0:1789302807.053971:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:3.0:1789302807.053982:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371140992:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302807.054037:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302807.054118:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371140992:12345-0@lo:13:osp-pre-1-0.0 Request processed in 138us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302807.054125:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1789302807.054268:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266c700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371140992:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302807.185343:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302807.567562:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371141120:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302807.567677:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302807.567884:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371141120 00000100:00100000:1.0:1789302807.567895:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371141120 00000100:00100000:1.0:1789302807.567967:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 187 00000100:00100000:1.0:1789302807.567979:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371141120:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302807.568033:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302807.584958:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371141120:12345-0@lo:13:osp-pre-0-0.0 Request processed in 16965us (17265us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302807.584977:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 187 00000100:00100000:0.0:1789302807.588866:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266f480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371141120:0@lo:13:osp-pre-0-0.0 00010000:00010000:3.0:1789302808.208048:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302809.231305:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302810.252824:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302810.445824:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302810.445839:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302810.446839:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 32 count 1 00000040:00080000:1.0:1789302810.446852:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302811.276861:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302811.661883:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371141248:0@lo:400:kworker.0 00000100:00100000:2.0:1789302811.661957:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302811.662130:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371141632:0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.662163:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302811.662199:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371141504:0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.662224:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302811.662238:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302811.662244:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371141376:0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.662263:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302811.662750:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371141504 00000100:00100000:0.0:1789302811.662759:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371141504 00000100:00100000:0.0:1789302811.662808:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371141376 00000100:00100000:0.0:1789302811.662811:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371141376 00000100:00100000:0.0:1789302811.662832:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1789302811.662840:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371141504:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.662895:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371141504:12345-0@lo:400:kworker.0 Request processed in 55us (670us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302811.662902:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:0.0:1789302811.662918:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:0.0:1789302811.662924:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371141376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.662953:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371141376:12345-0@lo:400:kworker.0 Request processed in 29us (689us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302811.662958:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371141248 00000100:00100000:0.0:1789302811.662959:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00000100:00100000:1.0:1789302811.662988:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371141248 00000100:00100000:0.0:1789302811.663015:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371141376:0@lo:400:kworker.0 00000100:00100000:1.0:1789302811.663023:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 578 00000100:00100000:0.0:1789302811.663029:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371141504:0@lo:400:kworker.0 00000100:00100000:1.0:1789302811.663035:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371141248:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.663076:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371141632 00000100:00100000:0.0:1789302811.666739:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371141632 00000100:00100000:0.0:1789302811.666820:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1789302811.666831:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371141632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.666922:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371141632:12345-0@lo:400:kworker.0 Request processed in 93us (4758us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302811.666928:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:0.0:1789302811.666994:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371141632:0@lo:400:kworker.0 00000100:00080000:0.0:1789302811.670264:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302811.670273:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302811.676245:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302811.676264:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371141760:0@lo:400:kworker.0 00000100:00100000:3.0:1789302811.676336:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302811.676417:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371141888:0@lo:400:kworker.0 00000100:00100000:3.0:1789302811.676447:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302811.676519:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371141760 00000100:00100000:3.0:1789302811.676526:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371141760 00000100:00100000:3.0:1789302811.676581:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:3.0:1789302811.676590:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371141760:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302811.676642:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371141760:12345-0@lo:400:kworker.0 Request processed in 53us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.676649:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00080000:3.0:1789302811.676669:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302811.676694:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266fb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371141760:0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.677423:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371141888 00000100:00100000:0.0:1789302811.677430:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371141888 00000100:00100000:0.0:1789302811.677481:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:0.0:1789302811.677489:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371141888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302811.677558:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371141888:12345-0@lo:400:kworker.0 Request processed in 70us (1109us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302811.677564:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1789302811.678014:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371141888:0@lo:400:kworker.0 00000100:00080000:3.0:1789302811.678037:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302811.681181:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371141248:12345-0@lo:400:kworker.0 Request processed in 18140us (19214us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302811.681196:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 578 00000100:00100000:2.0:1789302811.682145:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371141248:0@lo:400:kworker.0 00000100:00100000:2.0:1789302811.698769:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302811.702582:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350403968 00000100:00100000:0.0:1789302811.702596:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350403968 00000100:00100000:0.0:1789302811.702746:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1166 00000100:00100000:0.0:1789302811.702759:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29460:x1876219350403968:12345-192.168.203.11@tcp:101:diff.0 00000100:00080000:3.0:1789302811.702764:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302811.702781:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302811.702802:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a086d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302811.702902:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5f0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302811.702917:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302811.702928:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302811.702940:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1789302811.703104:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a086d expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302811.703123:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5f0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a086d expref: 98 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302811.703135:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302811.703137:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302811.703140:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044000, rc 0) 00000100:00100000:0.0:1789302811.703207:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29460:x1876219350403968:12345-192.168.203.11@tcp:101:diff.0 Request processed in 451us (4447us total) trans 4294968066 rc 0/0 00000100:00100000:0.0:1789302811.703214:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1166 00000100:00100000:2.0:1789302811.706105:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302811.706394:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350404352 00000100:00100000:3.0:1789302811.706404:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350404352 00000100:00100000:3.0:1789302811.706468:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 51 00000100:00100000:3.0:1789302811.706479:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:29460:x1876219350404352:12345-192.168.203.11@tcp:103:diff.0 00010000:00010000:3.0:1789302811.706494:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789302811.706536:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e5cd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0866 expref: 98 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.706559:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e4d8 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0820 expref: 97 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.706570:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302811.706575:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302811.706617:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29460:x1876219350404352:12345-192.168.203.11@tcp:103:diff.0 Request processed in 140us (521us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.706624:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 51 00000100:00100000:1.0:1789302811.707551:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302811.707823:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350404096 00000100:00100000:3.0:1789302811.707830:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350404096 00000100:00100000:3.0:1789302811.707878:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 52 00000100:00100000:3.0:1789302811.707886:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2348:x1876219350404096:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302811.707896:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302811.707911:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e5f0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a086d expref: 96 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.707926:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302811.707935:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302811.707970:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:2348:x1876219350404096:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 85us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.707985:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 52 00000100:00100000:2.0:1789302811.716874:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302811.717668:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350404480 00000100:00100000:0.0:1789302811.717678:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350404480 00000100:00100000:0.0:1789302811.717736:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1167 00000100:00100000:0.0:1789302811.717746:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29460:x1876219350404480:12345-192.168.203.11@tcp:58:diff.0 00010000:00010000:0.0:1789302811.717841:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302811.717858:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302811.717870:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789302811.718180:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].33, 136 off12680 00000100:00100000:0.0:1789302811.718258:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29460:x1876219350404480:12345-192.168.203.11@tcp:58:diff.0 Request processed in 514us (1404us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302811.718266:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1167 00000100:00100000:2.0:1789302811.722109:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302811.722182:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350404608 00000100:00100000:3.0:1789302811.722191:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350404608 00000100:00100000:3.0:1789302811.722249:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1168 00000100:00100000:3.0:1789302811.722259:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29460:x1876219350404608:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:3.0:1789302811.722279:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302811.722300:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0874 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.722363:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.722375:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0874 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.722406:0:5844:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.722421:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0874 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.722434:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f03501100) 00010000:00010000:3.0:1789302811.722438:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302811.722442:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0874 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302811.722450:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1c00, rc 0) 00000100:00100000:3.0:1789302811.722535:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29460:x1876219350404608:12345-192.168.203.11@tcp:101:diff.0 Request processed in 277us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.722543:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1168 00000100:00100000:1.0:1789302811.915751:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302811.915804:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302811.915841:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302811.915891:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302811.915911:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302811.915935:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302811.916089:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350404864 00000100:00100000:3.0:1789302811.916103:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350404864 00000100:00100000:3.0:1789302811.916186:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350405376 00000100:00100000:3.0:1789302811.916189:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350405376 00000100:00100000:3.0:1789302811.916224:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350404992 00000100:00100000:3.0:1789302811.916227:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350404992 00000100:00100000:3.0:1789302811.916241:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350405248 00000100:00100000:3.0:1789302811.916244:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350405248 00000100:00100000:3.0:1789302811.916262:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:3.0:1789302811.916274:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350404864:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302811.916369:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350404864:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 96us (633us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.916378:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:3.0:1789302811.916398:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:3.0:1789302811.916404:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350405376:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302811.916434:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350405376:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (592us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.916441:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:3.0:1789302811.916455:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 440 00000100:00100000:3.0:1789302811.916460:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350404992:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302811.916499:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350404992:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (587us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.916537:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 440 00000100:00100000:3.0:1789302811.916550:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:3.0:1789302811.916557:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350405248:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302811.916582:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350405248:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (647us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302811.916588:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:2.0:1789302811.916943:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350404736 00000100:00100000:2.0:1789302811.916949:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350404736 00000100:00100000:2.0:1789302811.917002:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 579 00000100:00100000:2.0:1789302811.917010:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350404736:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302811.917043:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350404736:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (1151us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302811.917050:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 579 00000100:00100000:2.0:1789302811.917085:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350405120 00000100:00100000:2.0:1789302811.917090:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350405120 00000100:00100000:2.0:1789302811.917132:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00100000:2.0:1789302811.917139:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2348:x1876219350405120:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302811.917169:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2348:x1876219350405120:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (1365us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302811.917174:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00080000:1.0:1789302811.917586:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302811.917592:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302811.917595:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302812.177942:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302812.177983:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371142016:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302812.178080:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302812.179248:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371142016 00000100:00100000:3.0:1789302812.179263:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371142016 00000100:00100000:3.0:1789302812.179328:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:3.0:1789302812.179340:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371142016:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302812.179392:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 79888384 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302812.179468:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371142016:12345-0@lo:13:osp-pre-1-0.0 Request processed in 128us (1387us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.179476:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:3.0:1789302812.179558:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371142016:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302812.323735:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302812.492784:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302812.492792:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302812.492832:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 34 count 2 00000040:00080000:1.0:1789302812.492840:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302812.492946:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371142144:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302812.493043:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a540 going to sleep for 11 seconds 00000100:00100000:2.0:1789302812.499491:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371142144:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302812.501049:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.501295:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350405504 00000100:00100000:3.0:1789302812.501312:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350405504 00000100:00100000:3.0:1789302812.501401:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1169 00000100:00100000:3.0:1789302812.501414:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29461:x1876219350405504:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302812.501456:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302812.501638:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29461:x1876219350405504:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 225us (593us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.501646:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1169 00000100:00100000:1.0:1789302812.519992:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.520149:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350405632 00000100:00100000:0.0:1789302812.520165:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350405632 00000100:00100000:0.0:1789302812.520251:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1170 00000100:00100000:0.0:1789302812.520263:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29461:x1876219350405632:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1789302812.520336:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520352:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520362:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520377:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520383:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520389:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520409:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520417:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520423:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302812.520458:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00000004:00200000:0.0:1789302812.520463:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302812.520473:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520488:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520495:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520521:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520527:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302812.520531:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520538:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520547:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302812.520550:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302812.520554:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520559:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e60c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520575:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520581:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520587:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520593:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302812.520596:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520602:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520610:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x329c2aeb (ffff960f03501000) 00010000:00010000:0.0:1789302812.520612:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302812.520615:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520629:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e613 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x329c2aeb bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520644:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520650:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520655:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520659:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302812.520662:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520667:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520678:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f03501100) 00010000:00010000:0.0:1789302812.520680:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302812.520683:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.520688:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e61a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302812.520766:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29461:x1876219350405632:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 505us (798us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.520774:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1170 00000100:00100000:3.0:1789302812.525920:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.526055:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350405888 00000100:00100000:0.0:1789302812.526080:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350405888 00000100:00100000:0.0:1789302812.526161:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1171 00000100:00100000:0.0:1789302812.526172:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29461:x1876219350405888:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302812.526192:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302812.526212:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e621 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a087b expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526315:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526329:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526341:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302812.526351:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x20:0x0]: returning size 158 00010000:00010000:0.0:1789302812.526446:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526455:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526461:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526467:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302812.526471:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526478:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526490:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f03501100) 00010000:00010000:0.0:1789302812.526493:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302812.526498:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526794:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e628 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x20:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526833:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e621 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a087b expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526842:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302812.526845:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302812.526849:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e621 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a087b expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.526865:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044600, rc 0) 00000100:00100000:0.0:1789302812.527099:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29461:x1876219350405888:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 928us (1193us total) trans 4294968067 rc 301/301 00000100:00100000:0.0:1789302812.527107:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1171 00000100:00100000:3.0:1789302812.535064:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.535115:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350406016 00000100:00100000:3.0:1789302812.535128:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350406016 00000100:00100000:3.0:1789302812.535193:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1172 00000100:00100000:3.0:1789302812.535206:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29461:x1876219350406016:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302812.535265:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x20:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302812.535410:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29461:x1876219350406016:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 206us (374us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.535422:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1172 00000100:00100000:1.0:1789302812.555679:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.556482:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350406272 00000100:00100000:1.0:1789302812.556498:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350406272 00000100:00100000:1.0:1789302812.557926:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1173 00000100:00100000:1.0:1789302812.557941:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:29461:x1876219350406272:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302812.557993:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302812.558022:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e636 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0882 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.563084:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e63d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.563111:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e63d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.563126:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e63d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302812.563139:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x22:0x0]: returning size 0 00010000:00010000:1.0:1789302812.563223:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e636 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0882 expref: 48 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.563246:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e63d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0882 expref: 47 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.563256:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302812.563260:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302812.563264:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:1.0:1789302812.563364:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29461:x1876219350406272:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 5425us (7695us total) trans 4294968068 rc 0/0 00000100:00100000:1.0:1789302812.563375:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1173 00000100:00100000:2.0:1789302812.582880:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.582976:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350406528 00000100:00100000:0.0:1789302812.582994:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350406528 00000100:00100000:0.0:1789302812.583088:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1174 00000100:00100000:0.0:1789302812.583102:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29461:x1876219350406528:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302812.583181:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302812.583218:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e63d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0882 expref: 47 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583242:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:0.0:1789302812.583256:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302812.583288:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583304:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583315:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583332:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583340:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583348:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583870:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583880:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583887:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583894:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302812.583901:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583910:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583922:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:0.0:1789302812.583925:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302812.583931:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583940:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e64b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x46:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583955:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583961:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583978:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583983:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302812.583986:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.583992:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.584000:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x8c4a2c72 (ffff960f03501000) 00010000:00010000:0.0:1789302812.584003:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302812.584006:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.584011:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e652 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x46:0x0].0x8c4a2c72 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302812.584191:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29461:x1876219350406528:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1092us (1316us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302812.584200:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1174 00000100:00100000:0.0:1789302812.588648:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302812.589078:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350406656 00000100:00100000:2.0:1789302812.589087:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350406656 00000100:00100000:2.0:1789302812.589156:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1175 00000100:00100000:2.0:1789302812.589178:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29461:x1876219350406656:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302812.589195:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302812.589216:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e659 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0889 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.589249:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.589262:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.589273:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.589355:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e659 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.589371:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.589379:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:2.0:1789302812.589382:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302812.589385:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:2.0:1789302812.589443:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29461:x1876219350406656:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 277us (800us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302812.589456:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1175 00000100:00100000:1.0:1789302812.593211:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.593571:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350406912 00000100:00100000:1.0:1789302812.593610:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350406912 00000100:00100000:1.0:1789302812.593685:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1176 00000100:00100000:1.0:1789302812.593707:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29461:x1876219350406912:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302812.593828:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.593850:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.593936:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302812.594649:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003526e19b reply state 00000000b257ec28 transno 4294968069 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302812.594663:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302812.599544:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29461:x1876219350406912:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 5827us (6326us total) trans 4294968069 rc 0/0 00000100:00100000:1.0:1789302812.599557:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1176 00010000:00010000:1.0:1789302812.611316:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.611334:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.611342:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.611348:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302812.611353:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.611376:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.611393:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:1.0:1789302812.611397:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302812.611402:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.611409:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e667 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.611764:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.613733:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350407040 00000100:00100000:1.0:1789302812.613954:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350407040 00000100:00100000:1.0:1789302812.614021:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1177 00000100:00100000:1.0:1789302812.614030:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:29461:x1876219350407040:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302812.614634:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302812.614656:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0890 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.614717:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 49 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.614730:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 49 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.614739:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:1.0:1789302812.614742:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302812.614745:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6e00, rc 0) 00000100:00100000:1.0:1789302812.614814:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29461:x1876219350407040:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 784us (3053us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302812.614820:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1177 00000100:00100000:3.0:1789302812.630432:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.630627:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350407168 00000100:00100000:0.0:1789302812.630638:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350407168 00000100:00100000:0.0:1789302812.630695:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 234 00000100:00100000:0.0:1789302812.630713:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076edc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29461:x1876219350407168:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:0.0:1789302812.630832:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076edc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29461:x1876219350407168:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 121us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.630843:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 234 00000100:00100000:1.0:1789302812.644879:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.645010:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350407424 00000100:00100000:0.0:1789302812.645021:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350407424 00000100:00100000:0.0:1789302812.645081:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 235 00000100:00100000:0.0:1789302812.645091:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:29461:x1876219350407424:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302812.645300:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29461:x1876219350407424:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 210us (428us total) trans 4294968070 rc 0/0 00000100:00100000:0.0:1789302812.645308:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 235 00000100:00100000:0.0:1789302812.656956:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.657047:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350407680 00000100:00100000:1.0:1789302812.657056:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350407680 00000100:00100000:1.0:1789302812.657113:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1178 00000100:00100000:1.0:1789302812.657124:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29461:x1876219350407680:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302812.657236:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29461:x1876219350407680:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 114us (288us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302812.657244:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1178 00000100:00100000:3.0:1789302812.672983:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.674151:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350407936 00000100:00100000:1.0:1789302812.674162:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350407936 00000100:00100000:1.0:1789302812.674228:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1179 00000100:00100000:1.0:1789302812.674238:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29461:x1876219350407936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302812.674256:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302812.674276:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e675 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0897 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.674368:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.674382:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 51 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.674390:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 4/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.674437:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 51 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.674450:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 51 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302812.674474:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371142272:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302812.674579:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80540 going to sleep for 11 seconds 00000100:00100000:2.0:1789302812.688666:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302812.688687:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371142400:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302812.688749:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302812.688795:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371142400 00000100:00100000:2.0:1789302812.688802:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371142400 00000100:00100000:2.0:1789302812.688843:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789302812.688851:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371142400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302812.688881:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302812.688922:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371142400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 72us (173us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302812.688929:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 189 00000100:00100000:2.0:1789302812.688980:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fb8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371142400:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302812.690892:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.691536:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350408064 00000100:00100000:3.0:1789302812.691544:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350408064 00010000:00010000:3.0:1789302812.691580:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 52 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.691614:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 351 00000100:00100000:3.0:1789302812.691642:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350408064:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302812.691655:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302812.691662:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 52 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.691675:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 52 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.691694:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:3.0:1789302812.691700:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.691710:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.691718:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.691741:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302812.691787:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350408064:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 165us (902us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.691793:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 351 00010000:00010000:2.0:1789302812.704281:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e66e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0890 expref: 51 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302812.704304:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdd880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371142272:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302812.704330:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.704775:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.704784:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302812.704804:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x22:0x0]: returning size 0 00000004:00080000:2.0:1789302812.705112:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x18:0x0], index 1: 0 00010000:00010000:2.0:1789302812.705242:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705251:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705258:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705263:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302812.705268:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705275:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705289:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:2.0:1789302812.705292:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302812.705297:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705303:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e67c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705353:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e675 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0897 expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705363:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x46:0x0].0x0 (ffff960f03501200) 00010000:00010000:2.0:1789302812.705366:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302812.705369:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e675 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x46:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0897 expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.705392:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7c00, rc 0) 00000100:00100000:2.0:1789302812.705454:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29461:x1876219350407936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 31216us (32487us total) trans 4294968071 rc 301/301 00000100:00100000:2.0:1789302812.705463:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1179 00000100:00100000:0.0:1789302812.718305:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.718364:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350408320 00000100:00100000:0.0:1789302812.718373:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350408320 00000100:00100000:0.0:1789302812.718417:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 236 00000100:00100000:0.0:1789302812.718426:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:29461:x1876219350408320:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302812.721332:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:29461:x1876219350408320:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 2899us (3028us total) trans 4294968072 rc 0/0 00000100:00100000:0.0:1789302812.721344:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 236 00000100:00100000:2.0:1789302812.728863:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.731212:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350408448 00000100:00100000:0.0:1789302812.731225:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350408448 00000100:00100000:0.0:1789302812.731425:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1180 00000100:00100000:0.0:1789302812.731436:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:29461:x1876219350408448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302812.731460:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302812.731481:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e68a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a089e expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.733190:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e691 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.733209:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e691 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.733221:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e691 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302812.733256:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x22:0x0]: returning size 0 00000004:00200000:0.0:1789302812.733260:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302812.733278:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e691 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.733308:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e68a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a089e expref: 53 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.733327:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e691 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a089e expref: 52 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.733335:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:0.0:1789302812.733338:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1789302812.733342:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044000, rc 0) 00000100:00100000:0.0:1789302812.733576:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29461:x1876219350408448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2140us (4720us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.733585:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1180 00000100:00100000:0.0:1789302812.745262:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.745299:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350408576 00000100:00100000:0.0:1789302812.745308:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350408576 00000100:00100000:0.0:1789302812.745359:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 318 00000100:00100000:0.0:1789302812.745369:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350408576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302812.745388:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302812.745408:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08a5 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302812.745435:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x18:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302812.745453:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=649s v=(0 0) 00010000:00010000:0.0:1789302812.745458:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302812.745467:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.745483:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.745496:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045000, rc 0) 00000100:00100000:0.0:1789302812.745562:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350408576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 194us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.745569:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 318 00000100:00100000:0.0:1789302812.753242:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.753361:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350408832 00000100:00100000:3.0:1789302812.753370:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350408832 00000100:00100000:3.0:1789302812.753431:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1181 00000100:00100000:3.0:1789302812.753441:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29461:x1876219350408832:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302812.753556:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29461:x1876219350408832:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 116us (321us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.753565:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1181 00000100:00100000:0.0:1789302812.765867:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.766811:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350408960 00000100:00100000:3.0:1789302812.766822:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350408960 00000100:00100000:3.0:1789302812.767067:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 319 00000100:00100000:3.0:1789302812.767078:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:29461:x1876219350408960:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302812.767095:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302812.767112:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 2/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08ac expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.767141:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.767150:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302812.767154:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302812.767158:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 4/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.767190:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.767203:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 11 pid: 18578 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.767217:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371142528:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302812.767306:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66b40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302812.772096:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371142528:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302812.772131:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 3/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.772150:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:1.0:1789302812.772205:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:29461:x1876219350408960:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 5130us (6344us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302812.772214:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 319 00000100:00100000:3.0:1789302812.774027:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.774070:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350409088 00000100:00100000:3.0:1789302812.774079:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350409088 00010000:00010000:3.0:1789302812.774118:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 11 pid: 18578 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.774154:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 352 00000100:00100000:3.0:1789302812.774163:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09751850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350409088:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302812.774177:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302812.774201:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 11 pid: 18578 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.774217:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 11 pid: 18578 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.774242:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e698 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08a5 expref: 11 pid: 18578 timeout: 749 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.774260:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302812.774263:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302812.774271:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.774280:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.774317:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.774331:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371142656:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302812.774394:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302812.780741:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371142656:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302812.780765:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302812.780821:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09751850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350409088:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 6658us (6800us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.780832:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 352 00000100:00100000:3.0:1789302812.782202:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.782273:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350409216 00000100:00100000:0.0:1789302812.782281:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350409216 00002000:00100000:0.0:1789302812.782326:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x18:0x0] (0->157) req@ffff960f120c8700 x1876219350409216/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:388/0 lens 648/0 e 0 to 0 dl 1789302823 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302812.782345:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c8700 with x1876219350409216 ext(0->157) 00002000:00010000:0.0:1789302812.782349:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f120c8700 00000100:00100000:0.0:1789302812.782381:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 105 00000100:00100000:0.0:1789302812.782390:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350409216:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:0.0:1789302812.782871:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350409216:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 480us (672us total) trans 4294967363 rc 0/0 00000100:00100000:0.0:1789302812.782879:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 105 00002000:00100000:0.0:1789302812.782886:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x18:0x0] (0->157) req@ffff960f120c8700 x1876219350409216/t4294967363(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:388/0 lens 648/448 e 0 to 0 dl 1789302823 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302812.782895:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120c8700 with x1876219350409216 ext(0->157) 00002000:00010000:0.0:1789302812.782898:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f120c8700 00000100:00100000:0.0:1789302812.785545:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302812.785697:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350409344 00000100:00100000:2.0:1789302812.785706:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350409344 00000100:00100000:2.0:1789302812.785786:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1182 00000100:00100000:2.0:1789302812.785796:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:29461:x1876219350409344:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1789302812.831804:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968072 is committed 00000001:00080000:3.0:1789302812.831826:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968066 is committed 00000100:00100000:2.0:1789302812.832147:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29461:x1876219350409344:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 46348us (46603us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302812.832158:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1182 00000100:00100000:3.0:1789302812.836427:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.836525:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350409472 00000100:00100000:1.0:1789302812.836534:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350409472 00000100:00100000:1.0:1789302812.836594:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:1.0:1789302812.836605:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350409472:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000001:00080000:2.0:1789302812.877282:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967363 is committed 00000100:00100000:1.0:1789302812.877463:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350409472:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 40848us (41032us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302812.877475:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:0.0:1789302812.882214:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.882433:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350409728 00000100:00100000:1.0:1789302812.882732:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350409728 00000100:00100000:1.0:1789302812.882809:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1183 00000100:00100000:1.0:1789302812.882820:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29461:x1876219350409728:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302812.882867:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302812.882889:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e691 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a089e expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.882906:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:1.0:1789302812.882911:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302812.882936:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.882948:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.882957:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302812.883135:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000092f5c8ac reply state 00000000ba950654 transno 4294968073 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302812.883144:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302812.883230:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29461:x1876219350409728:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 412us (1023us total) trans 4294968073 rc 0/0 00000100:00100000:1.0:1789302812.883237:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1183 00000100:00100000:0.0:1789302812.888267:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302812.889972:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.889995:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.890002:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.890009:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302812.890014:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.890021:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.890038:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:3.0:1789302812.890042:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302812.890046:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.890052:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6a6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302812.890371:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350409856 00000100:00100000:2.0:1789302812.890382:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350409856 00000100:00100000:2.0:1789302812.890794:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1184 00000100:00100000:2.0:1789302812.890806:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29461:x1876219350409856:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302812.890831:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302812.890849:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 2/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08b3 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.890904:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.890917:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.890951:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:2.0:1789302812.890954:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302812.890956:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302812.891042:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29461:x1876219350409856:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 232us (2777us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302812.891049:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1184 00000100:00100000:3.0:1789302812.894209:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302812.894379:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350409984 00000100:00100000:1.0:1789302812.894390:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350409984 00000100:00100000:1.0:1789302812.894442:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:1.0:1789302812.894463:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350409984:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1789302812.894759:0:18578:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:1.0:1789302812.894763:0:18578:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb mtime from disk: 0 -> 1789302788 00002000:00010000:1.0:1789302812.894766:0:18578:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb atime from disk: 0 -> 1789302788 00002000:00010000:1.0:1789302812.894768:0:18578:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb ctime from disk: 0 -> 1789302788 00002000:00010000:1.0:1789302812.894770:0:18578:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x18:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1789302812.894823:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350409984:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 373us (621us total) trans 4294967364 rc 0/0 00000100:00100000:1.0:1789302812.894831:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:3.0:1789302812.906984:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302812.907083:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350410240 00000100:00100000:2.0:1789302812.907093:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350410240 00000100:00100000:2.0:1789302812.907150:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1185 00000100:00100000:2.0:1789302812.907159:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:29461:x1876219350410240:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1789302812.907220:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.907237:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.907246:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.907278:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.907291:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 47 pid: 5843 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302812.907305:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371142784:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302812.907408:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33ff9cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302812.910673:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302812.910834:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371142784:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302812.910862:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302812.910952:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350410368 00000100:00100000:0.0:1789302812.910960:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350410368 00010000:00010000:0.0:1789302812.911007:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 48 pid: 5843 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302812.911050:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 353 00000100:00100000:0.0:1789302812.911059:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2347:x1876219350410368:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302812.911074:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302812.911080:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 48 pid: 5843 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.911092:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 48 pid: 5843 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.911112:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e660 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x22:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0889 expref: 48 pid: 5843 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.911128:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:0.0:1789302812.911132:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.911142:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.911150:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.911176:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302812.911237:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350410368:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 179us (577us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.911245:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 353 00010000:00010000:2.0:1789302812.911310:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911318:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911346:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911354:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911361:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911367:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302812.911372:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911379:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911404:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:2.0:1789302812.911408:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302812.911413:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302812.911439:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e6b4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302812.911565:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29461:x1876219350410240:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 4344us (4539us total) trans 0 rc -22/-22 00000100:00100000:2.0:1789302812.911574:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1185 00000100:00100000:3.0:1789302812.915177:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.915387:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350410496 00000100:00100000:0.0:1789302812.915407:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350410496 00000100:00100000:0.0:1789302812.915465:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 322 00000100:00100000:0.0:1789302812.915474:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350410496:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302812.915948:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/1,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.915985:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.915994:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302812.915998:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302812.916002:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 4/1,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.916036:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.916052:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 11 pid: 6695 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302812.916065:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371142912:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302812.916143:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335899c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302812.933067:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.933440:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350410624 00000100:00100000:3.0:1789302812.933460:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350410624 00010000:00010000:3.0:1789302812.933779:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 12 pid: 6695 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.933821:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 354 00000100:00100000:3.0:1789302812.933833:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350410624:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302812.933848:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302812.933869:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 12 pid: 6695 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.933883:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 12 pid: 6695 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.933906:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302812.933910:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302812.933916:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.933926:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.933946:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302812.934102:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350410624:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 271us (1044us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.934110:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 354 00010000:00010000:0.0:1789302812.934255:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e69f lrc: 0/0,0 mode: --/PW res: [0x280000400:0x18:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ac expref: 11 pid: 6695 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302812.934271:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371142912:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302812.934293:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934366:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934375:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934407:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934416:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934424:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934431:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302812.934854:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934866:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934879:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.934887:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e6bb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302812.934940:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350410496:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 19466us (19770us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302812.934948:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 322 00000100:00100000:1.0:1789302812.942450:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.942778:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350410880 00000100:00100000:0.0:1789302812.942789:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350410880 00000100:00100000:0.0:1789302812.942847:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1186 00000100:00100000:0.0:1789302812.942857:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:29461:x1876219350410880:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302812.942998:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.943015:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.943029:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.943226:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.943238:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.943245:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.943278:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.943291:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 47 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302812.943304:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371143040:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302812.943392:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335899c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302812.949093:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371143040:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302812.949145:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302812.949256:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.957106:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350411008 00000100:00100000:3.0:1789302812.957120:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350411008 00010000:00010000:3.0:1789302812.957167:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 48 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.957221:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 355 00000100:00100000:3.0:1789302812.957231:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350411008:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302812.957246:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302812.957253:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 48 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.957267:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 48 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.957288:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e6ad lrc: 0/0,0 mode: --/CR res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08b3 expref: 48 pid: 6919 timeout: 749 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.957306:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:3.0:1789302812.957312:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.957321:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.957329:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.957367:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302812.957430:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350411008:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 201us (8183us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302812.957438:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 355 00010000:00010000:1.0:1789302812.957786:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302812.957795:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789302812.957989:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968074 00000001:00080000:1.0:1789302812.958141:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968075 00000001:00080000:1.0:1789302812.958189:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968076 00000100:00100000:1.0:1789302812.958266:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371143168:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789302812.958779:0:5845:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302812.958825:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302812.958955:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371143168 00000100:00100000:0.0:1789302812.958974:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371143168 00000100:00100000:0.0:1789302812.959027:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:0.0:1789302812.959043:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371143168:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1789302812.959128:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371143168:12345-0@lo:1000:mdt00_002.0 Request processed in 94us (348us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.959135:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000100:00100000:3.0:1789302812.959269:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdc000 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371143168:0@lo:1000:mdt00_002.0 00000001:00080000:3.0:1789302812.959888:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968077 00000100:00100000:3.0:1789302812.960144:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371143296:0@lo:1000:mdt00_002.0 00000100:00100000:3.0:1789302812.960183:0:5845:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302812.960212:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc669c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302812.960876:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371143296 00000100:00100000:1.0:1789302812.960885:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371143296 00000100:00100000:1.0:1789302812.960933:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:1.0:1789302812.960941:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5845:x1876219371143296:12345-0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1789302812.961174:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5845:x1876219371143296:12345-0@lo:1000:mdt00_002.0 Request processed in 233us (989us total) trans 4294967365 rc 0/0 00000100:00100000:1.0:1789302812.961182:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:3.0:1789302812.961460:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-mdtlov_UUID:5845:1876219371143296:0@lo:1000:mdt00_002.0 00000001:00080000:3.0:1789302812.961569:0:5845:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968078 00010000:00010000:3.0:1789302812.961599:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961620:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961629:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961635:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302812.961640:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961648:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961662:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f02d72700) 00010000:00010000:3.0:1789302812.961665:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302812.961670:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961676:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e6c9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x22:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961707:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961714:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961724:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961730:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302812.961732:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961738:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961746:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f03501100) 00010000:00010000:3.0:1789302812.961748:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302812.961755:0:5845:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0874 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961766:0:5845:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0874 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961775:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0874 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.961803:0:5845:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1c00/0x64fe43095184e605 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0874 expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.961815:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371143424:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302812.961869:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc669c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302812.970943:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.971296:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350411136 00000100:00100000:0.0:1789302812.971306:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350411136 00000100:00100000:0.0:1789302812.971374:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 237 00000100:00100000:0.0:1789302812.971384:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29460:x1876219350411136:12345-192.168.203.11@tcp:49:diff.0 00000100:00100000:0.0:1789302812.971544:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29460:x1876219350411136:12345-192.168.203.11@tcp:49:diff.0 Request processed in 160us (609us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.971552:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 237 00000100:00100000:3.0:1789302812.971901:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8a80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371143424:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302812.971932:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.971943:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e5fe lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.971988:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972003:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972015:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972025:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972032:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972038:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972045:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302812.972049:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972057:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972070:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f03501100) 00010000:00010000:3.0:1789302812.972073:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302812.972078:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972084:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e6d0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x20:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972095:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972102:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972109:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972114:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302812.972118:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972124:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972132:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x20:0x0].0x0 (ffff960f03501100) 00010000:00010000:3.0:1789302812.972135:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302812.972137:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.972144:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e6c2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x20:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302812.972327:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:29461:x1876219350410880:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 29470us (29885us total) trans 4294968079 rc 0/0 00000100:00100000:3.0:1789302812.972339:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1186 00000020:00080000:3.0:1789302812.972344:0:5845:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:2.0:1789302812.979155:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302812.979164:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302812.979695:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 2 00000040:00080000:2.0:1789302812.979705:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302812.983107:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302812.983162:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.983174:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350411264 00000100:00100000:0.0:1789302812.983184:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350411264 00000100:00100000:0.0:1789302812.983242:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 323 00000100:00100000:0.0:1789302812.983253:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:29460:x1876219350411264:12345-192.168.203.11@tcp:101:diff.0 00010000:00010000:0.0:1789302812.983270:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302812.983291:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000746e2d77/0x64fe43095184e6d7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08ba expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302812.983316:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x18:0x0] initial LVB size: 158, mtime: 0x6aa69804, atime: 0x6aa69804, ctime: 0x6aa69804, blocks: 0xb 00010000:00010000:0.0:1789302812.983339:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=649s v=(0 0) 00010000:00010000:0.0:1789302812.983343:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x18:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302812.983366:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000746e2d77/0x64fe43095184e6d7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ba expref: 17 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.983381:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000746e2d77/0x64fe43095184e6d7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x18:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08ba expref: 17 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302812.983393:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045200, rc 0) 00000100:00100000:0.0:1789302812.983452:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:29460:x1876219350411264:12345-192.168.203.11@tcp:101:diff.0 Request processed in 201us (354us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302812.983460:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 323 00000100:00100000:3.0:1789302812.983594:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350411520 00000100:00100000:3.0:1789302812.983599:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350411520 00000100:00100000:3.0:1789302812.983639:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 238 00000100:00100000:3.0:1789302812.983647:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:29461:x1876219350411520:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1789302812.984267:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x22:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1789302812.984381:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968080 00010000:00010000:3.0:1789302812.984421:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 3/0,1 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984435:0:12976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984444:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984452:0:12976:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984459:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984466:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984486:0:12976:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302812.984567:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:29461:x1876219350411520:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 920us (1404us total) trans 4294968081 rc 0/0 00000100:00100000:3.0:1789302812.984578:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 238 00000020:00080000:3.0:1789302812.984582:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:3.0:1789302812.984601:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302812.984607:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984613:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302812.984618:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984625:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984636:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x22:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302812.984638:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302812.984643:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 1/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984649:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e6de lrc: 0/0,0 mode: --/PW res: [0x200000402:0x22:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302812.984664:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302812.984666:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302812.984668:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302812.991070:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302812.993285:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350411648 00000100:00100000:3.0:1789302812.993298:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350411648 00002000:00100000:3.0:1789302812.993357:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x18:0x0] (0->4095) req@ffff960f3fcdf100 x1876219350411648/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:388/0 lens 488/0 e 0 to 0 dl 1789302823 ref 1 fl New:/600/ffffffff rc 0/-1 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302812.993368:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdf100 with x1876219350411648 ext(0->4095) 00002000:00010000:3.0:1789302812.993373:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fcdf100 00000100:00100000:3.0:1789302812.993417:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 106 00000100:00100000:3.0:1789302812.993428:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350411648:12345-192.168.203.11@tcp:3:diff.0 00000100:00100000:3.0:1789302812.993702:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350411648:12345-192.168.203.11@tcp:3:diff.0 Request processed in 275us (2639us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302812.993709:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 106 00002000:00100000:3.0:1789302812.993716:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x18:0x0] (0->4095) req@ffff960f3fcdf100 x1876219350411648/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:388/0 lens 488/4536 e 0 to 0 dl 1789302823 ref 1 fl Complete:/600/0 rc 4096/4096 job:'diff.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302812.993723:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdf100 with x1876219350411648 ext(0->4095) 00002000:00010000:3.0:1789302812.993727:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fcdf100 00000100:00100000:2.0:1789302812.999568:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302812.999899:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350411904 00000100:00100000:0.0:1789302812.999908:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350411904 00000100:00100000:0.0:1789302812.999977:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 239 00000100:00100000:0.0:1789302812.999989:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:29460:x1876219350411904:12345-192.168.203.11@tcp:35:diff.0 00000100:00100000:0.0:1789302813.000248:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:29460:x1876219350411904:12345-192.168.203.11@tcp:35:diff.0 Request processed in 260us (688us total) trans 4294968082 rc 0/0 00000100:00100000:0.0:1789302813.000257:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 239 00010000:00010000:3.0:1789302813.342244:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302814.348924:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302814.594763:0:21008:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302814.594772:0:21008:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302814.594809:0:21008:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 34 count 2 00000040:00080000:0.0:1789302814.594815:0:21008:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302815.372966:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302816.396926:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302816.783701:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371143552:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.783771:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302816.783814:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371144064:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.783840:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302816.783879:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302816.783885:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371143936:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.783911:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302816.783954:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371143680:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.783979:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302816.783996:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371144192:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.784016:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302816.784033:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302816.784039:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371143808:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.784059:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302816.784092:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371143936 00000100:00100000:3.0:1789302816.784100:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371143936 00000100:00100000:3.0:1789302816.784147:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144192 00000100:00100000:3.0:1789302816.784150:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144192 00000100:00100000:3.0:1789302816.784170:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:3.0:1789302816.784179:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371143936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.784226:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371143936:12345-0@lo:400:kworker.0 Request processed in 49us (314us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302816.784233:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:3.0:1789302816.784247:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1789302816.784253:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371144192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.784284:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371144192:12345-0@lo:400:kworker.0 Request processed in 31us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302816.784290:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:3.0:1789302816.784305:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144064 00000100:00100000:3.0:1789302816.784308:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144064 00000100:00100000:3.0:1789302816.784331:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371143680 00000100:00100000:3.0:1789302816.784334:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371143680 00000100:00100000:3.0:1789302816.784349:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371143808 00000100:00100000:3.0:1789302816.784351:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371143808 00000100:00100000:3.0:1789302816.784365:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:3.0:1789302816.784370:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371144064:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.784399:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371144064:12345-0@lo:400:kworker.0 Request processed in 28us (558us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302816.784405:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:3.0:1789302816.784416:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:3.0:1789302816.784422:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371143680:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302816.784550:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371143552 00000100:00100000:1.0:1789302816.784554:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371143552 00000100:00100000:1.0:1789302816.784583:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 580 00000100:00100000:1.0:1789302816.784590:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371143552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302816.784674:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:2.0:1789302816.784682:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371143808:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.784973:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038db100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371143680:12345-0@lo:400:kworker.0 Request processed in 551us (992us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302816.784979:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00080000:3.0:1789302816.785001:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302816.785005:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302816.785006:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302816.785748:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371143552:12345-0@lo:400:kworker.0 Request processed in 1156us (1974us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302816.785755:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 580 00000100:00100000:2.0:1789302816.785874:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371143808:12345-0@lo:400:kworker.0 Request processed in 1192us (1812us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302816.785882:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:3.0:1789302816.786364:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371143936:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.786376:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371143552:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.786386:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371144064:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.786424:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371143808:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.786435:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371143680:0@lo:400:kworker.0 00000100:00100000:3.0:1789302816.787163:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371144192:0@lo:400:kworker.0 00000100:00100000:1.0:1789302817.060312:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302817.060356:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302817.060397:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302817.060424:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302817.060461:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302817.060485:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302817.060544:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302817.060573:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412288 00000100:00100000:1.0:1789302817.060582:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412288 00000100:00100000:1.0:1789302817.060639:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412800 00000100:00100000:1.0:1789302817.060642:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412800 00000100:00100000:1.0:1789302817.060674:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412416 00000100:00100000:1.0:1789302817.060676:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412416 00000100:00100000:1.0:1789302817.060689:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412672 00000100:00100000:1.0:1789302817.060691:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412672 00000100:00100000:1.0:1789302817.060705:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 444 00000100:00100000:1.0:1789302817.060714:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350412288:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302817.060778:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2347:x1876219350412288:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 65us (421us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302817.060786:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 444 00000100:00100000:1.0:1789302817.060802:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 445 00000100:00100000:1.0:1789302817.060808:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350412800:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302817.060830:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350412800:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (406us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302817.060837:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 445 00000100:00100000:1.0:1789302817.060848:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:1.0:1789302817.060854:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350412416:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302817.060874:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350412416:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302817.060880:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:1.0:1789302817.060891:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00100000:1.0:1789302817.060897:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350412672:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302817.060919:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350412672:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (375us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302817.060924:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00080000:1.0:1789302817.060974:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302817.060979:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302817.060989:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412544 00000100:00100000:1.0:1789302817.060992:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412544 00000100:00100000:1.0:1789302817.061029:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412160 00000100:00100000:1.0:1789302817.061033:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412160 00000100:00100000:1.0:1789302817.061050:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:1.0:1789302817.061057:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2347:x1876219350412544:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302817.061090:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2347:x1876219350412544:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (692us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302817.061095:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:1.0:1789302817.061107:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 442 00000100:00100000:1.0:1789302817.061113:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350412160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302817.061139:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:2347:x1876219350412160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (677us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302817.061145:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 442 00000100:00080000:1.0:1789302817.061158:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302817.062180:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412032 00000100:00100000:0.0:1789302817.062189:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412032 00000100:00100000:0.0:1789302817.062233:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 581 00000100:00100000:0.0:1789302817.062241:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350412032:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302817.062290:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350412032:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 51us (1990us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302817.062296:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 581 00000100:00100000:1.0:1789302817.292756:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302817.292783:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371144320:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302817.292867:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302817.293229:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144320 00000100:00100000:2.0:1789302817.293238:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144320 00000100:00100000:2.0:1789302817.293292:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:2.0:1789302817.293301:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371144320:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302817.293332:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302817.293393:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371144320:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (524us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302817.293417:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:1.0:1789302817.293655:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371144320:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302817.421461:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302817.810763:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371144448:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302817.810869:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302817.812391:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144448 00000100:00100000:0.0:1789302817.812412:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144448 00000100:00100000:0.0:1789302817.812476:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 191 00000100:00100000:0.0:1789302817.812488:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371144448:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302817.812561:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302817.814285:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371144448:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1789us (3405us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302817.814300:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 191 00000100:00100000:1.0:1789302817.818251:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371144448:0@lo:13:osp-pre-0-0.0 00000001:00080000:3.0:1789302817.869911:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968082 is committed 00000001:00080000:3.0:1789302817.869940:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968074 is committed 00000001:00080000:3.0:1789302817.869950:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968077 is committed 00000001:00080000:3.0:1789302817.869970:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968081 is committed 00000020:00080000:3.0:1789302817.869988:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f3fc66e40 refcount 2 committed 0 result 0 batchid 24 00000020:00080000:3.0:1789302817.869993:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f41c9c900 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f3fc66c00 00000020:00080000:3.0:1789302817.869998:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f41c9c000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f09f5bc00 00000020:00080000:3.0:1789302817.870028:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 24 committed batchid 23 00000020:00080000:3.0:1789302817.870373:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 23 00000001:00080000:1.0:1789302818.374861:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967365 is committed 00000001:00080000:1.0:1789302818.374882:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967364 is committed 00010000:00010000:3.0:1789302818.445804:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302819.469173:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302820.173067:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302820.173082:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302820.173134:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 34 count 2 00000040:00080000:3.0:1789302820.173143:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302820.437238:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302820.437737:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350412928 00000100:00100000:1.0:1789302820.437756:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350412928 00000100:00100000:1.0:1789302820.437832:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1187 00000100:00080000:0.0:1789302820.437840:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302820.437846:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29346:x1876219350412928:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1789302820.438082:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:29346:x1876219350412928:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 239us (858us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302820.438090:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1187 00010000:00010000:3.0:1789302820.497812:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302821.518155:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302821.906728:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371144832:0@lo:400:kworker.0 00000100:00100000:0.0:1789302821.906948:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302821.907030:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789302821.907037:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371144704:0@lo:400:kworker.0 00000100:00100000:0.0:1789302821.907064:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302821.907086:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371145216:0@lo:400:kworker.0 00000100:00100000:0.0:1789302821.907112:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302821.907148:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144832 00000100:00100000:0.0:1789302821.907157:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144832 00000100:00100000:0.0:1789302821.907221:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144704 00000100:00100000:0.0:1789302821.907224:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144704 00000100:00100000:0.0:1789302821.907249:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:0.0:1789302821.907258:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371144832:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302821.907321:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371144832:12345-0@lo:400:kworker.0 Request processed in 64us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302821.907330:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:0.0:1789302821.907351:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:3.0:1789302821.907594:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371144576:0@lo:400:kworker.0 00000100:00100000:3.0:1789302821.907639:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302821.907674:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371145088:0@lo:400:kworker.0 00000100:00100000:3.0:1789302821.907699:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302821.907719:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789302821.907724:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371144960:0@lo:400:kworker.0 00000100:00100000:3.0:1789302821.907746:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302821.907947:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144576 00000100:00100000:1.0:1789302821.907954:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144576 00000100:00100000:1.0:1789302821.907999:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 582 00000100:00100000:1.0:1789302821.908007:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371144576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302821.908055:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371144576:12345-0@lo:400:kworker.0 Request processed in 49us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302821.908062:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 582 00000100:00100000:1.0:1789302821.908104:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371145216 00000100:00100000:1.0:1789302821.908107:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371145216 00000100:00100000:1.0:1789302821.908135:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371144960 00000100:00100000:1.0:1789302821.908138:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371144960 00000100:00100000:1.0:1789302821.908158:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1789302821.908165:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371145216:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302821.908197:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371145216:12345-0@lo:400:kworker.0 Request processed in 33us (1084us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302821.908203:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1789302821.908213:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1789302821.908220:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371144960:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302821.908246:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371144960:12345-0@lo:400:kworker.0 Request processed in 27us (499us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302821.908251:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1789302821.908286:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371145088 00000100:00100000:1.0:1789302821.908290:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371145088 00000100:00100000:0.0:1789302821.908694:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371144704:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302821.908741:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371144704:12345-0@lo:400:kworker.0 Request processed in 53us (1675us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302821.909033:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1789302821.909044:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371145088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302821.909081:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371145088:12345-0@lo:400:kworker.0 Request processed in 40us (1381us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302821.909087:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1789302821.909154:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371144960:0@lo:400:kworker.0 00000100:00100000:1.0:1789302821.909168:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371144576:0@lo:400:kworker.0 00000100:00100000:1.0:1789302821.909179:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371145088:0@lo:400:kworker.0 00000100:00080000:1.0:1789302821.910983:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302821.910995:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302821.910999:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302821.911072:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:0.0:1789302821.911177:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371144704:0@lo:400:kworker.0 00000100:00100000:0.0:1789302821.911194:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967365 gen 2 00000004:00080000:0.0:1789302821.911201:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967365 committed transno 4294967365 00000100:00100000:0.0:1789302821.911229:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371145216:0@lo:400:kworker.0 00000100:00100000:0.0:1789302821.911243:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371144832:0@lo:400:kworker.0 00000100:00100000:3.0:1789302822.165757:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.165815:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.165844:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.165865:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.165887:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.165915:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.165946:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.165982:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413056 00000100:00100000:3.0:1789302822.165994:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413056 00000100:00100000:3.0:1789302822.166055:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00100000:3.0:1789302822.166068:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350413056:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302822.166750:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350413056:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 679us (1003us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.166761:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00100000:3.0:1789302822.166831:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413312 00000100:00100000:3.0:1789302822.166838:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413312 00000100:00100000:3.0:1789302822.166888:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413824 00000100:00100000:3.0:1789302822.166892:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413824 00000100:00100000:3.0:1789302822.166923:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413440 00000100:00100000:3.0:1789302822.166926:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413440 00000100:00100000:3.0:1789302822.166942:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413696 00000100:00100000:3.0:1789302822.166944:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413696 00000100:00100000:3.0:1789302822.166962:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 450 00000100:00100000:3.0:1789302822.166970:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350413312:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302822.167008:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350413312:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 39us (1193us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.167014:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 450 00000100:00100000:3.0:1789302822.167026:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:3.0:1789302822.167032:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350413824:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302822.167055:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350413824:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (1189us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.167062:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:3.0:1789302822.167072:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00100000:3.0:1789302822.167079:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350413440:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302822.167100:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350413440:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (1184us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.167105:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00100000:3.0:1789302822.167115:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 453 00000100:00100000:3.0:1789302822.167122:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350413696:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302822.167144:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350413696:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (1198us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.167150:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 453 00000100:00100000:3.0:1789302822.167173:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413568 00000100:00100000:3.0:1789302822.167177:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413568 00000100:00100000:3.0:1789302822.167207:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413184 00000100:00100000:3.0:1789302822.167211:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413184 00000100:00100000:3.0:1789302822.167242:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:3.0:1789302822.167248:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2346:x1876219350413568:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302822.168876:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2346:x1876219350413568:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1621us (3024us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.168887:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:3.0:1789302822.168948:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00100000:3.0:1789302822.168962:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:2346:x1876219350413184:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302822.168991:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:2346:x1876219350413184:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (3104us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.168997:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00080000:3.0:1789302822.170949:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302822.170962:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302822.170966:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:02000400:2.0:1789302822.377947:0:21207:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12p: implicit restore of a file on copytool mount point ========================================================== 08:33:40 (1789302820) 00000100:00100000:3.0:1789302822.412873:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302822.412915:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371145344:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302822.413023:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302822.414393:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371145344 00000100:00100000:0.0:1789302822.414410:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371145344 00000100:00100000:0.0:1789302822.414495:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:0.0:1789302822.414523:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371145344:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302822.414574:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302822.414674:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371145344:12345-0@lo:13:osp-pre-1-0.0 Request processed in 154us (1650us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302822.414680:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:0.0:1789302822.414747:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e0e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371145344:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302822.540712:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302822.665673:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302822.665973:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350413952 00000100:00100000:2.0:1789302822.665988:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350413952 00000100:00100000:2.0:1789302822.666064:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1188 00000100:00100000:2.0:1789302822.666078:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:30075:x1876219350413952:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302822.666261:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:30075:x1876219350413952:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 186us (602us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302822.666269:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1188 00000100:00100000:3.0:1789302822.695913:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302822.696134:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350414080 00000100:00100000:1.0:1789302822.696150:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350414080 00000100:00100000:1.0:1789302822.696891:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1189 00000100:00100000:1.0:1789302822.696918:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:30076:x1876219350414080:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302822.696948:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302822.696991:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e6e5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08c1 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697088:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697109:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697124:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697162:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697170:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697177:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697484:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697493:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697524:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697533:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302822.697538:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697547:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697562:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302822.697566:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.697572:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697579:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e6ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697601:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697607:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697614:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697620:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302822.697623:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697630:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697638:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff960f2c94d000) 00010000:00010000:1.0:1789302822.697641:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.697644:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697650:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e6f3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697692:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e6e5 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08c1 expref: 96 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697705:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302822.697707:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.697718:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e6e5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08c1 expref: 96 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.697729:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302822.698117:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:30076:x1876219350414080:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1201us (2224us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302822.703255:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1189 00000100:00100000:0.0:1789302822.703459:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302822.703618:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350414336 00000100:00100000:1.0:1789302822.703631:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350414336 00000100:00100000:1.0:1789302822.703682:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1190 00000100:00100000:1.0:1789302822.703693:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:30076:x1876219350414336:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:1.0:1789302822.703780:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302822.703826:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e318 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a077f expref: 95 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.703848:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302822.703854:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302822.704164:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.704182:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.704193:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.704236:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 43 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.704268:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 43 pid: 5843 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302822.704287:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371145472:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302822.704612:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3358aa80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302822.707287:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.707383:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350414464 00000100:00100000:0.0:1789302822.707395:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350414464 00010000:00010000:0.0:1789302822.707459:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 44 pid: 5843 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302822.707532:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 356 00000100:00100000:0.0:1789302822.707545:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350414464:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302822.707567:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302822.707576:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 44 pid: 5843 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.707592:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 44 pid: 5843 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.707624:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302822.707630:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.707641:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.707650:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.707674:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302822.707740:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:2347:x1876219350414464:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 198us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302822.707749:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 356 00010000:00010000:1.0:1789302822.708787:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e357 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0794 expref: 43 pid: 5843 timeout: 759 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302822.708827:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7800 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371145472:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302822.708858:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.708935:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.708944:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.708996:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.709009:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.709019:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302822.710455:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 16384 00010000:00010000:0.0:1789302822.710481:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.710492:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.710528:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.710535:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.710540:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.710549:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.710569:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302822.710573:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.710579:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.710589:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e6fa lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302822.710620:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002990cd5d reply state 000000002100c4ae transno 4294968083 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302822.710631:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302822.710784:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:30076:x1876219350414336:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 7087us (7331us total) trans 4294968083 rc 0/0 00000100:00100000:0.0:1789302822.710795:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1190 00010000:00010000:1.0:1789302822.715340:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.715365:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.716254:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.716267:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302822.716274:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.716284:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.716334:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302822.716338:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.716346:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.716353:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e701 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302822.724962:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.725184:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350414592 00000100:00100000:3.0:1789302822.725199:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350414592 00000100:00100000:3.0:1789302822.725281:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1191 00000100:00100000:3.0:1789302822.725294:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+95:30077:x1876219350414592:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302822.725331:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302822.725383:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e708 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08c8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.725480:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e70f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.725497:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e70f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.725571:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e70f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302822.725626:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302822.725631:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302822.725655:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e70f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.725685:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184e708 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08c8 expref: 97 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.725707:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e70f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08c8 expref: 96 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.725717:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302822.725721:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302822.725726:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302822.725809:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:30077:x1876219350414592:12345-192.168.203.11@tcp:101:bash.0 Request processed in 519us (864us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.725817:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1191 00000100:00100000:0.0:1789302822.735186:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302822.735562:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350414720 00000100:00100000:1.0:1789302822.735577:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350414720 00000100:00100000:1.0:1789302822.735654:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1192 00000100:00100000:1.0:1789302822.735666:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:30077:x1876219350414720:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302822.735696:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302822.735725:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e716 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08cf expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.735815:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.735834:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.735848:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.735865:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.735872:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.735907:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736043:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e72b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736052:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e72b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736059:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e72b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302822.736095:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 16384 00000004:00200000:1.0:1789302822.736100:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302822.736123:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e72b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736137:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736144:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736151:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736158:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302822.736163:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736173:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736195:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302822.736199:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.736205:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736213:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e71d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736229:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736237:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736242:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736247:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302822.736639:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736651:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736660:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff960f2c94d000) 00010000:00010000:1.0:1789302822.736662:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.736666:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736671:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e724 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736727:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184e716 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08cf expref: 98 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736743:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6e00/0x64fe43095184e72b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08cf expref: 97 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.736751:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:1.0:1789302822.736754:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302822.736757:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6e00, rc 0) 00000100:00100000:1.0:1789302822.736868:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30077:x1876219350414720:12345-192.168.203.11@tcp:101:bash.0 Request processed in 1203us (1696us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302822.736877:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1192 00000100:00100000:1.0:1789302822.750351:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.750478:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350414848 00000100:00100000:0.0:1789302822.750493:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350414848 00000100:00100000:0.0:1789302822.750602:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1193 00000100:00100000:0.0:1789302822.750614:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30077:x1876219350414848:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302822.750642:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302822.750671:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e732 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08d6 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750763:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750782:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750797:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750814:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750823:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750830:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750943:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750953:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750960:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750968:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.750973:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750982:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.750997:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.751000:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.751006:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751013:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e739 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751027:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751034:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751040:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751045:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.751047:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751053:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751059:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f09ae1100) 00010000:00010000:0.0:1789302822.751062:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.751065:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751071:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e740 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751110:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e732 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08d6 expref: 98 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751123:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.751125:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.751129:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e732 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08d6 expref: 98 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.751146:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045a00, rc 0) 00000100:00100000:0.0:1789302822.751232:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30077:x1876219350414848:12345-192.168.203.11@tcp:101:bash.0 Request processed in 621us (905us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302822.751241:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1193 00000100:00100000:2.0:1789302822.772716:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.772803:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350414976 00000100:00100000:0.0:1789302822.772819:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350414976 00000100:00100000:0.0:1789302822.772954:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1194 00000100:00100000:0.0:1789302822.772968:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+43:30078:x1876219350414976:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302822.772998:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302822.773026:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e747 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08dd expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.773126:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.773149:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.773165:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302822.773217:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302822.773222:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302822.773248:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.773285:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e747 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 45 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.773314:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 44 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.773323:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302822.773327:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302822.773332:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044000, rc 0) 00000100:00100000:0.0:1789302822.773428:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:30078:x1876219350414976:12345-192.168.203.11@tcp:101:rm.0 Request processed in 463us (730us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302822.773436:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1194 00000100:00100000:3.0:1789302822.775634:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.775734:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350415104 00000100:00100000:0.0:1789302822.775749:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350415104 00000100:00100000:0.0:1789302822.775816:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1195 00000100:00100000:0.0:1789302822.775830:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:30078:x1876219350415104:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302822.775862:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302822.775889:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e755 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08e4 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.775986:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776005:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776027:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776046:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776054:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776061:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776196:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e76a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776206:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e76a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776213:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e76a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302822.776248:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 16384 00000004:00200000:0.0:1789302822.776252:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302822.776356:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e76a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776603:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776613:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776622:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776631:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.776635:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776643:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776658:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302822.776661:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.776668:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776675:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e75c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776693:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776700:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776709:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776714:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.776717:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776723:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776730:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x55434c71 (ffff960f09ae1100) 00010000:00010000:0.0:1789302822.776732:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.776735:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776745:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e763 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x55434c71 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776792:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e755 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08e4 expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776807:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045e00/0x64fe43095184e76a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08e4 expref: 45 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.776814:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.776817:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302822.776819:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045e00, rc 0) 00000100:00100000:0.0:1789302822.779006:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30078:x1876219350415104:12345-192.168.203.11@tcp:101:rm.0 Request processed in 3160us (3365us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302822.779029:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1195 00000100:00100000:0.0:1789302822.784447:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302822.784654:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350415232 00000100:00100000:2.0:1789302822.784669:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350415232 00000100:00100000:2.0:1789302822.784744:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1196 00000100:00100000:2.0:1789302822.784757:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30078:x1876219350415232:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302822.784786:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302822.784812:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e771 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08eb expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785044:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785064:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785099:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785116:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785124:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785131:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785202:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785211:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785219:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785225:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302822.785231:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785239:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785253:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:2.0:1789302822.785256:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302822.785263:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785276:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e778 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785291:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785297:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785302:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785307:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302822.785310:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785315:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785322:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f02d72700) 00010000:00010000:2.0:1789302822.785325:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302822.785328:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785332:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e77f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785372:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e771 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08eb expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785395:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:2.0:1789302822.785398:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302822.785402:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e771 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08eb expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.785413:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302822.785533:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30078:x1876219350415232:12345-192.168.203.11@tcp:101:rm.0 Request processed in 745us (1067us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302822.785542:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1196 00000100:00100000:2.0:1789302822.791618:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.794377:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350415360 00000100:00100000:0.0:1789302822.794399:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350415360 00000100:00100000:0.0:1789302822.795087:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1197 00000100:00100000:0.0:1789302822.795107:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30078:x1876219350415360:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302822.795140:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302822.795170:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e786 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08f2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795277:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795301:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795315:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795333:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795340:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795347:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795421:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795429:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795436:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795442:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.795447:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795456:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795470:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.795474:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.795480:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795486:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e78d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795525:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795533:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795539:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795544:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.795547:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795553:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795561:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f09ae1100) 00010000:00010000:0.0:1789302822.795564:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.795567:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795573:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e794 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795616:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e786 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08f2 expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795628:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.795630:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.795634:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e786 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08f2 expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.795644:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045c00, rc 0) 00000100:00100000:0.0:1789302822.795730:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30078:x1876219350415360:12345-192.168.203.11@tcp:101:rm.0 Request processed in 630us (4127us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302822.807074:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1197 00000100:00100000:3.0:1789302822.819275:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.821224:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350415488 00000100:00100000:0.0:1789302822.821249:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350415488 00000100:00100000:0.0:1789302822.821349:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1198 00000100:00100000:0.0:1789302822.821362:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30079:x1876219350415488:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302822.821393:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302822.821426:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e79b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a08f9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821556:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821575:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821591:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821608:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821616:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821623:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821693:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821702:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821709:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821715:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.821720:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821729:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821743:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.821746:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.821752:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821759:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7a2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821773:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821779:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821786:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821791:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302822.821794:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821799:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821805:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f09ae1100) 00010000:00010000:0.0:1789302822.821807:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.821809:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821815:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7a9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821852:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e79b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08f9 expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821862:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.821875:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302822.821879:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e79b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08f9 expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.821893:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045c00, rc 0) 00000100:00100000:0.0:1789302822.821980:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30079:x1876219350415488:12345-192.168.203.11@tcp:101:cp.0 Request processed in 621us (2720us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302822.821988:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1198 00000100:00100000:1.0:1789302822.825926:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302822.826086:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350415616 00000100:00100000:2.0:1789302822.826100:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350415616 00000100:00100000:2.0:1789302822.826181:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1199 00000100:00100000:2.0:1789302822.826195:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30079:x1876219350415616:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302822.826227:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302822.826256:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e7b0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0900 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826353:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826371:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826383:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826421:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826429:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826435:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826548:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826556:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826562:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826569:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302822.826574:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826583:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826597:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:2.0:1789302822.826601:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302822.826607:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826614:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184e7b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826637:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826643:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826650:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826656:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302822.826659:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826665:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826673:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f02d72700) 00010000:00010000:2.0:1789302822.826676:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302822.826679:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826685:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184e7be lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826723:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e7b0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0900 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826733:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:2.0:1789302822.826735:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302822.826744:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184e7b0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0900 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.826754:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2200, rc 0) 00000100:00100000:2.0:1789302822.826830:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30079:x1876219350415616:12345-192.168.203.11@tcp:101:cp.0 Request processed in 638us (919us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302822.826838:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1199 00000100:00100000:2.0:1789302822.831311:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.831416:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350415872 00000100:00100000:0.0:1789302822.831430:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350415872 00000100:00100000:0.0:1789302822.831539:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1200 00000100:00100000:0.0:1789302822.831552:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30079:x1876219350415872:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302822.831582:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302822.831609:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e7c5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0907 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.831766:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.831784:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.831798:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.831815:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.831822:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.831828:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302822.832757:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x19:0x0], index 0: 0 00010000:00010000:1.0:1789302822.837966:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.837999:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838014:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838066:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838074:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838101:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838108:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302822.838113:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838121:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838135:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:1.0:1789302822.838138:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.838144:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838151:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e7cc lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302822.838170:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000647f4fa4 reply state 0000000006da61f3 transno 4294968084 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302822.838180:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302822.838234:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e7c5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838253:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.838260:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302822.838271:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302822.838274:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6800, rc 0) 00000100:00100000:1.0:1789302822.838446:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30079:x1876219350415872:12345-192.168.203.11@tcp:101:cp.0 Request processed in 6896us (7148us total) trans 4294968084 rc 0/0 00000100:00100000:1.0:1789302822.838456:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1200 00010000:00010000:1.0:1789302822.845494:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.845551:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.845560:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.845567:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302822.845574:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.845583:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.845605:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f09ae1100) 00010000:00010000:1.0:1789302822.845609:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302822.845616:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.845623:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e7d3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302822.846019:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.848593:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350416000 00000100:00100000:3.0:1789302822.848617:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350416000 00000100:00100000:3.0:1789302822.848776:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 324 00000100:00100000:3.0:1789302822.848791:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350416000:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302822.848832:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302822.848862:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a090e expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302822.848907:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302822.848936:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=659s v=(0 0) 00010000:00010000:3.0:1789302822.848941:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302822.848954:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.848985:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.849001:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302822.849102:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350416000:12345-192.168.203.11@tcp:101:cp.0 Request processed in 314us (3091us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.849110:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 324 00000100:00100000:0.0:1789302822.857317:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.861335:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350416128 00000100:00100000:3.0:1789302822.861362:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350416128 00000100:00100000:1.0:1789302822.861890:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:1.0:1789302822.861920:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:30079:x1876219350416128:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302822.861977:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302822.862006:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0915 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.862053:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.862063:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302822.862068:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302822.862074:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 4/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.862120:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.862136:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 11 pid: 6696 timeout: 759 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302822.862154:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371145600:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302822.862290:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3358aa80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302822.870392:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302822.871752:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371145600:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302822.871974:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 3/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.872011:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:1.0:1789302822.872129:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:30079:x1876219350416128:12345-192.168.203.11@tcp:101:cp.0 Request processed in 10215us (14825us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302822.872140:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:3.0:1789302822.874317:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350416256 00000100:00100000:3.0:1789302822.874337:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350416256 00010000:00010000:2.0:1789302822.874670:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 11 pid: 6696 timeout: 759 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302822.874772:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:2.0:1789302822.874785:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350416256:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302822.874821:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302822.874855:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 11 pid: 6696 timeout: 759 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.874872:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 11 pid: 6696 timeout: 759 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.874911:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e7e8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a090e expref: 11 pid: 6696 timeout: 759 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.874935:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302822.874940:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302822.874950:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.874990:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.875046:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302822.875065:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371145728:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302822.875197:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f142aac00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302822.877471:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371145728:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302822.877545:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302822.877676:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350416256:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 2892us (7300us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302822.877688:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:3.0:1789302822.878829:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.878983:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350416512 00000100:00100000:0.0:1789302822.878994:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350416512 00000100:00100000:0.0:1789302822.879060:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 240 00000100:00100000:0.0:1789302822.879070:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e9c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30079:x1876219350416512:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302822.881757:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e9c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30079:x1876219350416512:12345-192.168.203.11@tcp:35:cp.0 Request processed in 2666us (2915us total) trans 4294968085 rc 0/0 00000100:00100000:0.0:1789302822.881776:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 240 00000100:00100000:0.0:1789302822.926137:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.928122:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350416640 00000100:00100000:3.0:1789302822.928139:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350416640 00000100:00100000:3.0:1789302822.928225:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1201 00000100:00100000:3.0:1789302822.928238:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30082:x1876219350416640:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302822.928263:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302822.928291:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e7f6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a091c expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928379:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928398:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928411:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928427:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928435:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928442:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928791:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928803:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.928811:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302822.928871:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:3.0:1789302822.928877:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000100:00100000:0.0:1789302822.928988:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371145856:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302822.929090:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302822.929178:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371145856 00000100:00100000:0.0:1789302822.929185:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371145856 00000100:00100000:0.0:1789302822.929232:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789302822.929240:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371145856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302822.929276:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 81592320 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302822.929330:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371145856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302822.929338:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 193 00000100:00100000:0.0:1789302822.929395:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371145856:0@lo:13:osp-pre-0-0.0 00000004:00010000:3.0:1789302822.933212:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933241:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933253:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933261:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933269:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933276:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302822.933281:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933290:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933308:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:3.0:1789302822.933313:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302822.933319:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933325:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e7fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933340:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933346:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933352:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933357:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302822.933359:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933366:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933372:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f0cdf2a00) 00010000:00010000:3.0:1789302822.933375:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302822.933379:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933384:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e804 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933441:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e7f6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933461:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 98 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302822.933469:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:3.0:1789302822.933472:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302822.933474:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:3.0:1789302822.933634:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30082:x1876219350416640:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 5397us (7511us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302822.933645:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1201 00000100:00100000:2.0:1789302822.943628:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302822.943714:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350416768 00000100:00100000:3.0:1789302822.943728:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350416768 00000100:00100000:3.0:1789302822.943808:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:3.0:1789302822.943821:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350416768:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302822.943848:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302822.943874:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e812 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0923 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302822.943973:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371145984:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302822.944094:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fda9c0 going to sleep for 11 seconds 00002000:00010000:1.0:1789302822.949637:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb size: 0 -> 158 00002000:00010000:1.0:1789302822.949649:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb mtime: 0 -> 1789302822 00002000:00010000:1.0:1789302822.949652:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb atime: 0 -> 1789302822 00002000:00010000:1.0:1789302822.949654:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x19:0x0] updating lvb ctime: 0 -> 1789302822 00000100:00100000:1.0:1789302822.949690:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371145984:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302822.949742:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e812 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0923 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.949775:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184e812 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0923 expref: 12 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302822.949791:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:1.0:1789302822.949886:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2345:x1876219350416768:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 6066us (6273us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302822.949905:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:0.0:1789302822.956182:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302822.956234:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350416896 00000100:00100000:0.0:1789302822.956247:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350416896 00000100:00100000:0.0:1789302822.956319:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1202 00000100:00100000:0.0:1789302822.956331:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30082:x1876219350416896:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302822.956363:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302822.956392:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e819 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a092a expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.956487:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.956526:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.956541:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302822.956588:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 16384 00000004:00200000:0.0:1789302822.956593:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302822.956614:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.956649:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e819 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 100 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.956669:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302822.956679:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302822.956683:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302822.956688:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045000, rc 0) 00000100:00100000:0.0:1789302822.956770:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30082:x1876219350416896:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 441us (603us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302822.956778:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1202 00000100:00100000:1.0:1789302822.972380:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302822.972462:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350417024 00000100:00100000:2.0:1789302822.972481:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350417024 00000100:00100000:2.0:1789302822.972766:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 327 00000100:00100000:2.0:1789302822.972782:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350417024:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302822.973398:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302822.973430:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e827 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0931 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302822.973547:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371146112:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302822.973669:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f095bf300 going to sleep for 11 seconds 00000100:00100000:2.0:1789302822.993305:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371146112:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302822.993363:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e827 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0931 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.993393:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000810a9f4e/0x64fe43095184e827 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0931 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302822.993407:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3600, rc 0) 00000100:00100000:2.0:1789302822.993495:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2346:x1876219350417024:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 20716us (21137us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302822.993550:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 327 00000001:00080000:3.0:1789302822.994365:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968085 is committed 00000001:00080000:3.0:1789302822.994383:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968083 is committed 00000020:00080000:3.0:1789302822.994391:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 24 batchid updated 00000020:00080000:2.0:1789302822.994451:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 24 00000020:00080000:2.0:1789302822.994465:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 24 00000100:00100000:2.0:1789302823.004688:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302823.004812:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350417280 00000100:00100000:1.0:1789302823.004826:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350417280 00000100:00100000:1.0:1789302823.004921:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1203 00000100:00100000:1.0:1789302823.004936:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30082:x1876219350417280:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:1.0:1789302823.005668:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].34, 136 off12816 00000100:00100000:1.0:1789302823.005806:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30082:x1876219350417280:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 871us (1132us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302823.005828:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1203 00000040:00080000:2.0:1789302823.247430:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302823.247446:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302823.249535:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 35 count 3 00000040:00080000:2.0:1789302823.249559:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302823.251131:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371146240:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302823.251663:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f095bfcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302823.262521:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371146240:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302823.265186:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302823.265487:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350417408 00000100:00100000:1.0:1789302823.265577:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350417408 00000100:00100000:1.0:1789302823.265691:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1204 00000100:00100000:1.0:1789302823.265705:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30092:x1876219350417408:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302823.265760:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302823.265921:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30092:x1876219350417408:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 219us (748us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302823.265930:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1204 00000100:00100000:2.0:1789302823.270091:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302823.270461:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350417664 00000100:00100000:0.0:1789302823.270477:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350417664 00000100:00100000:0.0:1789302823.270657:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1205 00000100:00100000:0.0:1789302823.270669:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30092:x1876219350417664:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302823.270697:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302823.270726:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e82e lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0938 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.271193:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e835 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.271341:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e835 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.271452:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e835 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302823.271470:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:0.0:1789302823.271818:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e82e lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0938 expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.271844:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e835 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0938 expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.271992:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:0.0:1789302823.271996:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302823.272001:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045a00, rc 0) 00000100:00100000:0.0:1789302823.272149:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30092:x1876219350417664:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1481us (2071us total) trans 4294968086 rc 0/0 00000100:00100000:0.0:1789302823.272159:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1205 00000100:00100000:2.0:1789302823.283470:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302823.283539:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302823.283653:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350417920 00000100:00100000:1.0:1789302823.283667:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350417920 00000100:00100000:1.0:1789302823.283751:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 53 00000100:00100000:1.0:1789302823.283762:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2345:x1876219350417920:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302823.283784:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302823.283815:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e835 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0938 expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.283838:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302823.283844:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302823.283935:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2345:x1876219350417920:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 174us (394us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302823.283944:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 53 00000100:00100000:3.0:1789302823.284752:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350417792 00000100:00100000:3.0:1789302823.284763:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350417792 00000100:00100000:3.0:1789302823.284826:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 328 00000100:00100000:3.0:1789302823.284836:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350417792:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302823.284905:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.284929:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.284940:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x19:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302823.284945:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302823.284950:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.285056:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.285074:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 10 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302823.285092:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371146368:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302823.285193:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6300 going to sleep for 11 seconds 00000100:00100000:2.0:1789302823.302780:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302823.302993:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350418048 00000100:00100000:2.0:1789302823.303005:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350418048 00002000:00100000:2.0:1789302823.303074:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x19:0x0] (0->157) req@ffff960f25fbc000 x1876219350418048/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:399/0 lens 648/0 e 0 to 0 dl 1789302834 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302823.303088:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fbc000 with x1876219350418048 ext(0->157) 00010000:00010000:2.0:1789302823.303099:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 11 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.303118:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 11 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302823.303126:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f25fbc000 00000100:00100000:2.0:1789302823.303157:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 11 00000100:00100000:2.0:1789302823.303167:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350418048:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:2.0:1789302823.303863:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350418048:12345-192.168.203.11@tcp:4:cp.0 Request processed in 681us (1082us total) trans 4294967364 rc 0/0 00000100:00100000:2.0:1789302823.303874:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 11 00002000:00100000:2.0:1789302823.303883:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x19:0x0] (0->157) req@ffff960f25fbc000 x1876219350418048/t4294967364(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:399/0 lens 648/448 e 0 to 0 dl 1789302834 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302823.303900:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fbc000 with x1876219350418048 ext(0->157) 00010000:00010000:2.0:1789302823.303908:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 13 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.303923:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 13 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302823.303931:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f25fbc000 00000100:00100000:3.0:1789302823.306323:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371146368:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302823.306367:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302823.322362:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302823.322667:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350418176 00000100:00100000:3.0:1789302823.322682:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350418176 00010000:00010000:3.0:1789302823.322736:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 13 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302823.322788:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 358 00000100:00100000:3.0:1789302823.322801:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09750450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2346:x1876219350418176:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302823.322819:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302823.322848:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 13 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.322870:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 13 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.322911:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe43095184e7ef lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0915 expref: 13 pid: 18578 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.322931:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302823.322936:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302823.322944:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.322956:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.323002:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302823.323087:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09750450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350418176:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 288us (739us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302823.323096:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 358 00010000:00010000:1.0:1789302823.324779:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324824:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324874:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324885:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324903:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324911:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302823.324918:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324931:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324957:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.324966:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184e843 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302823.325085:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350417792:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 40247us (41626us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302823.325096:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 328 00000100:00100000:0.0:1789302823.347133:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302823.347174:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350418432 00000100:00100000:0.0:1789302823.347187:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350418432 00000100:00100000:0.0:1789302823.347254:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1206 00000100:00100000:0.0:1789302823.347265:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30092:x1876219350418432:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302823.347397:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30092:x1876219350418432:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 134us (278us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302823.347406:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1206 00000100:00100000:0.0:1789302823.359901:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302823.359990:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350418560 00000100:00100000:2.0:1789302823.360005:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350418560 00000100:00100000:2.0:1789302823.360084:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:2.0:1789302823.360096:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350418560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302823.360121:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302823.360150:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a093f expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302823.360185:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 158, mtime: 0x6aa69826, atime: 0x0, ctime: 0x6aa69826, blocks: 0x1 00010000:00010000:2.0:1789302823.360211:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=660s v=(0 0) 00010000:00010000:2.0:1789302823.360216:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302823.360227:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.360258:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.360276:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2a00, rc 0) 00000100:00100000:2.0:1789302823.360348:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350418560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 254us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302823.360354:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:3.0:1789302823.380843:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302823.393107:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350418816 00000100:00100000:1.0:1789302823.393132:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350418816 00000100:00100000:0.0:1789302823.393312:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1207 00000100:00100000:0.0:1789302823.393325:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30092:x1876219350418816:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302823.393485:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30092:x1876219350418816:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 161us (12656us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302823.393493:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1207 00000100:00100000:3.0:1789302823.402190:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302823.403271:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350418944 00000100:00100000:1.0:1789302823.403287:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350418944 00002000:00100000:1.0:1789302823.403358:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x19:0x0] (0->4095) req@ffff960f1e6ae300 x1876219350418944/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:399/0 lens 488/0 e 0 to 0 dl 1789302834 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302823.403371:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6ae300 with x1876219350418944 ext(0->4095) 00002000:00010000:1.0:1789302823.403379:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f1e6ae300 00000100:00100000:1.0:1789302823.403420:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 107 00000100:00100000:1.0:1789302823.403430:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350418944:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302823.404023:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350418944:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 590us (1841us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302823.404034:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 107 00002000:00100000:1.0:1789302823.404042:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x19:0x0] (0->4095) req@ffff960f1e6ae300 x1876219350418944/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:399/0 lens 488/4536 e 0 to 0 dl 1789302834 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302823.404051:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1e6ae300 with x1876219350418944 ext(0->4095) 00002000:00010000:1.0:1789302823.404055:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f1e6ae300 00000100:00100000:2.0:1789302823.415192:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302823.415367:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350419072 00000100:00100000:3.0:1789302823.415380:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350419072 00000100:00100000:3.0:1789302823.415458:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1208 00000100:00100000:3.0:1789302823.415472:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30092:x1876219350419072:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302823.415531:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302823.415559:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e851 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0946 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.415605:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.415623:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.415636:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.415768:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e851 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.415790:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302823.415801:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:3.0:1789302823.415804:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302823.415809:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1400, rc 0) 00000100:00100000:3.0:1789302823.416118:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30092:x1876219350419072:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 647us (939us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302823.416127:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1208 00000100:00100000:2.0:1789302823.421621:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302823.421716:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350419200 00000100:00100000:0.0:1789302823.421729:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350419200 00000100:00100000:0.0:1789302823.421799:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1209 00000100:00100000:0.0:1789302823.421812:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30092:x1876219350419200:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302823.421858:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302823.422004:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30092:x1876219350419200:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 195us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302823.422013:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1209 00000100:00100000:2.0:1789302823.427676:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302823.427727:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350419328 00000100:00100000:2.0:1789302823.427753:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350419328 00000100:00100000:2.0:1789302823.427823:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:2.0:1789302823.427835:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350419328:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302823.427901:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 3/1,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.427923:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302823.427929:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302823.427935:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.427953:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.427994:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.428003:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.428012:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.428019:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302823.428024:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.428045:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.428066:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.428074:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b237177/0x64fe43095184e85f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302823.428161:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350419328:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 327us (495us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302823.428169:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:2.0:1789302823.448715:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302823.448764:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350419584 00000100:00100000:2.0:1789302823.448777:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350419584 00000100:00100000:2.0:1789302823.448844:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1210 00000100:00100000:2.0:1789302823.448857:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30092:x1876219350419584:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302823.449033:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.449056:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.449066:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.449196:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302823.449213:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 48 pid: 5844 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302823.449231:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371146496:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302823.449418:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f095bfcc0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302823.457058:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302823.457948:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371146496:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302823.457989:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302823.465317:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350419712 00000100:00100000:0.0:1789302823.465342:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350419712 00010000:00010000:0.0:1789302823.465418:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 49 pid: 5844 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302823.465478:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 359 00000100:00100000:0.0:1789302823.465490:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350419712:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302823.465525:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302823.465535:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 49 pid: 5844 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.465550:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 49 pid: 5844 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.465580:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e858 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0946 expref: 49 pid: 5844 timeout: 760 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.465603:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:0.0:1789302823.465609:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.465621:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.466247:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302823.466297:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302823.466457:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350419712:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 967us (9409us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302823.466468:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 359 00010000:00010000:1.0:1789302823.467332:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467345:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467657:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467665:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467672:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467679:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302823.467684:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467692:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467708:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302823.467712:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302823.467717:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302823.467723:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e866 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302823.467923:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30092:x1876219350419584:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 19064us (19219us total) trans 4294968087 rc 0/0 00000100:00100000:1.0:1789302823.467950:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1210 00000040:00080000:1.0:1789302823.468093:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302823.468098:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302823.469960:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1789302823.469982:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302823.483181:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302823.483333:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350419968 00000100:00100000:0.0:1789302823.483348:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350419968 00000100:00100000:0.0:1789302823.483425:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 241 00000100:00100000:0.0:1789302823.483450:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30092:x1876219350419968:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302823.483908:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:30092:x1876219350419968:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 461us (739us total) trans 4294968088 rc 0/0 00000100:00100000:0.0:1789302823.483917:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 241 00010000:00010000:1.0:1789302823.565724:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302823.964933:0:21257:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302823.964947:0:21257:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302823.965001:0:21257:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 35 count 3 00000040:00080000:3.0:1789302823.965009:0:21257:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302824.139820:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.140281:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350420224 00000100:00100000:2.0:1789302824.140297:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350420224 00000100:00100000:2.0:1789302824.140388:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1211 00000100:00100000:2.0:1789302824.140401:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30098:x1876219350420224:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302824.140430:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302824.140460:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e86d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a094d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.140632:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e874 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.140652:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e874 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.140670:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e874 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302824.140684:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:2.0:1789302824.140860:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e86d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a094d expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.140883:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e874 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a094d expref: 100 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.140905:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:2.0:1789302824.140908:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302824.140912:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3a00, rc 0) 00000100:00100000:2.0:1789302824.141010:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30098:x1876219350420224:12345-192.168.203.11@tcp:101:cat.0 Request processed in 611us (1205us total) trans 4294968089 rc 0/0 00000100:00100000:2.0:1789302824.141017:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1211 00000100:00080000:3.0:1789302824.141599:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302824.148008:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.148148:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.148191:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350420480 00000100:00100000:3.0:1789302824.148204:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350420480 00000100:00100000:3.0:1789302824.148298:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:3.0:1789302824.148315:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350420480:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302824.148344:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302824.148370:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0954 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.148406:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302824.148411:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302824.148422:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 12 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.148447:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 12 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.148465:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302824.149782:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350420480:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1463us (1623us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.149795:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00080000:3.0:1789302824.149843:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302824.150677:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350420352 00000100:00100000:1.0:1789302824.150691:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350420352 00000100:00100000:1.0:1789302824.150767:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 54 00000100:00100000:1.0:1789302824.150778:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:2346:x1876219350420352:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302824.150797:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302824.150829:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184e874 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a094d expref: 100 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.150853:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302824.150859:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302824.150932:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:2346:x1876219350420352:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 156us (2939us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302824.150939:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 54 00000100:00100000:2.0:1789302824.160336:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.160422:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350420608 00000100:00100000:0.0:1789302824.160436:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350420608 00002000:00100000:0.0:1789302824.160562:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (0->4095) req@ffff960f3f70c000 x1876219350420608/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:400/0 lens 488/0 e 0 to 0 dl 1789302835 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302824.160576:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f70c000 with x1876219350420608 ext(0->4095) 00002000:00010000:0.0:1789302824.160584:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f70c000 00000100:00100000:0.0:1789302824.160617:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 108 00000100:00100000:0.0:1789302824.160628:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350420608:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:0.0:1789302824.160883:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350420608:12345-192.168.203.11@tcp:3:cat.0 Request processed in 256us (559us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302824.160902:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 108 00002000:00100000:0.0:1789302824.160910:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x19:0x0] (0->4095) req@ffff960f3f70c000 x1876219350420608/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:400/0 lens 488/4536 e 0 to 0 dl 1789302835 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302824.160936:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f70c000 with x1876219350420608 ext(0->4095) 00002000:00010000:0.0:1789302824.160940:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f70c000 00000100:00100000:2.0:1789302824.172369:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.179484:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350420864 00000100:00100000:0.0:1789302824.180241:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350420864 00000100:00100000:1.0:1789302824.180535:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:1.0:1789302824.180549:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30098:x1876219350420864:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302824.180903:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30098:x1876219350420864:12345-192.168.203.11@tcp:35:cat.0 Request processed in 339us (8533us total) trans 4294968090 rc 0/0 00000100:00100000:1.0:1789302824.180913:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 242 00000100:00100000:0.0:1789302824.218388:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.218701:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350421120 00000100:00100000:0.0:1789302824.218716:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350421120 00000100:00100000:0.0:1789302824.218898:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1212 00000100:00100000:0.0:1789302824.218912:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30099:x1876219350421120:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302824.218948:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302824.221124:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e889 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a095b expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.221724:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e890 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.221745:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e890 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.221759:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e890 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302824.221776:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:0.0:1789302824.222388:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e889 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a095b expref: 103 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.223241:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e890 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a095b expref: 102 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.223261:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:0.0:1789302824.223264:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302824.223268:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045c00, rc 0) 00000100:00100000:0.0:1789302824.223364:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30099:x1876219350421120:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4454us (4991us total) trans 4294968091 rc 0/0 00000100:00100000:0.0:1789302824.223375:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1212 00000100:00100000:3.0:1789302824.233673:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.233834:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350421248 00000100:00100000:0.0:1789302824.233847:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350421248 00000100:00100000:0.0:1789302824.233929:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 332 00000100:00100000:0.0:1789302824.233941:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350421248:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:0.0:1789302824.234018:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.234150:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.234229:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 12 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.234239:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302824.234244:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302824.234250:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 5/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.234295:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 12 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.234313:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 12 pid: 18578 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.234332:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371146624:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302824.234454:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.234465:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 11 pid: 13439 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.234476:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371146752:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302824.234525:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599780 going to sleep for 11 seconds 00000100:00100000:0.0:1789302824.253369:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.253400:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.253464:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdca80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371146624:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302824.253478:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdea00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371146752:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302824.253530:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.254961:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350421504 00000100:00100000:0.0:1789302824.254980:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350421504 00000100:00100000:1.0:1789302824.255529:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350421376 00000100:00100000:1.0:1789302824.255651:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350421376 00010000:00010000:1.0:1789302824.255703:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 13 pid: 18578 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.255751:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 360 00000100:00100000:1.0:1789302824.255763:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350421376:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302824.255783:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302824.255815:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 13 pid: 18578 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.255831:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 13 pid: 18578 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.255858:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f8011d48/0x64fe43095184e882 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0954 expref: 13 pid: 18578 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.255882:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302824.255993:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350421376:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 231us (2591us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302824.256003:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 360 00010000:00010000:0.0:1789302824.258325:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 12 pid: 13439 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.258410:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 361 00000100:00100000:0.0:1789302824.258424:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350421504:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302824.258444:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302824.258478:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 12 pid: 13439 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.258495:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 12 pid: 13439 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.258538:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097b61bb9/0x64fe43095184e84a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a093f expref: 12 pid: 13439 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.258830:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302824.258836:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302824.258844:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.258856:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.258909:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302824.258997:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350421504:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 577us (5643us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302824.259007:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 361 00010000:00010000:1.0:1789302824.259051:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259062:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259117:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259127:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259137:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259145:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302824.259150:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259159:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259183:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 2/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.259192:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184e89e lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.259252:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:2348:x1876219350421248:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 25314us (25595us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302824.259262:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 332 00000100:00080000:0.0:1789302824.259752:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302824.262670:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.263095:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350421760 00000100:00100000:2.0:1789302824.263110:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350421760 00000100:00100000:2.0:1789302824.263186:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 243 00000100:00100000:2.0:1789302824.263198:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30099:x1876219350421760:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:2.0:1789302824.263299:0:5847:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x23:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e890 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a095b expref: 102 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302824.263618:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968092 00000001:00080000:2.0:1789302824.264367:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968093 00010000:00010000:2.0:1789302824.264446:0:5847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.264467:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.264489:0:5847:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 105 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.264528:0:5847:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.264590:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 105 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.264608:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 105 pid: 6919 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.264624:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371146880:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302824.264752:0:5847:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.264762:0:5847:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 48 pid: 5843 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.264772:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371147008:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302824.264816:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f095bf3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302824.269137:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.269180:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371147008:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302824.269202:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70d180 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000_UUID:5847:1876219371146880:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302824.269240:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:0.0:1789302824.269257:0:5847:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.269387:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350421888 00000100:00100000:3.0:1789302824.269397:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350421888 00000100:00100000:1.0:1789302824.269452:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350422016 00000100:00100000:1.0:1789302824.269460:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350422016 00010000:00010000:1.0:1789302824.269492:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 106 pid: 6919 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.269551:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:1.0:1789302824.269630:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2348:x1876219350422016:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302824.269645:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 49 pid: 5843 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.269651:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302824.269663:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 106 pid: 6919 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.269677:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 363 00010000:00010000:1.0:1789302824.269682:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 106 pid: 6919 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.269711:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e80b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a091c expref: 106 pid: 6919 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.269731:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302824.269737:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.269748:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.269756:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302824.269826:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:2348:x1876219350422016:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 200us (585us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.269855:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350421888:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302824.269866:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302824.269872:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 49 pid: 5843 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:0.0:1789302824.269879:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302824.269894:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 49 pid: 5843 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.269914:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184e7da lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0907 expref: 49 pid: 5843 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.269931:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:3.0:1789302824.269935:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.269944:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.269964:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.270011:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 362 00010000:00010000:3.0:1789302824.270172:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302824.270199:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2347:x1876219350421888:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 349us (1078us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.270206:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 363 00010000:00010000:0.0:1789302824.270684:0:5847:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.270693:0:5847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302824.270852:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968094 00000001:00080000:1.0:1789302824.271822:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968095 00000001:00080000:1.0:1789302824.271920:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968096 00000100:00100000:1.0:1789302824.272104:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c5c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371147136:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:1.0:1789302824.272358:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302824.272449:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302824.272628:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371147136 00000100:00100000:0.0:1789302824.272638:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371147136 00000100:00100000:0.0:1789302824.272759:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:0.0:1789302824.272773:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371147136:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:0.0:1789302824.272901:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371147136:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 132us (543us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302824.272911:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000100:00100000:2.0:1789302824.272947:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c5c00 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371147136:0@lo:1000:mdt_rdpg00_001.0 00000100:00080000:1.0:1789302824.273368:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:2.0:1789302824.273413:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968097 00000100:00100000:2.0:1789302824.274132:0:5847:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371147264:0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1789302824.274196:0:5847:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302824.274233:0:5847:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f095bf3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302824.274310:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371147264 00000100:00100000:3.0:1789302824.274319:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371147264 00000100:00100000:3.0:1789302824.274423:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 54 00000100:00100000:3.0:1789302824.274434:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5847:x1876219371147264:12345-0@lo:1000:mdt_rdpg00_001.0 00000100:00100000:2.0:1789302824.274926:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5847:x1876219371147264:12345-0@lo:1000:mdt_rdpg00_001.0 Request processed in 493us (728us total) trans 4294967365 rc 0/0 00000100:00100000:2.0:1789302824.274935:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 54 00000100:00100000:1.0:1789302824.275021:0:5847:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f100 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_001:lustre-MDT0000-mdtlov_UUID:5847:1876219371147264:0@lo:1000:mdt_rdpg00_001.0 00010000:00010000:1.0:1789302824.275403:0:5847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.275417:0:5847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.275426:0:5847:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.275434:0:5847:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302824.275440:0:5847:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.275448:0:5847:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.275468:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302824.275471:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:1.0:1789302824.275477:0:5847:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.275483:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184e8a5 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302824.276737:0:5847:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x49:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1789302824.276901:0:5847:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].17, 64 off9328 00000004:00080000:1.0:1789302824.276920:0:5847:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 661287417109ns, next at 662287412587ns, rc = 0 00000001:00080000:1.0:1789302824.276933:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968098 00010000:00010000:1.0:1789302824.277136:0:5847:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302824.277141:0:5847:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1789302824.277154:0:5847:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e890 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a095b expref: 109 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.277448:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30099:x1876219350421760:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 14251us (14790us total) trans 4294968099 rc 0/0 00000100:00100000:1.0:1789302824.277459:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 243 00000020:00080000:1.0:1789302824.277466:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1789302824.348899:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302824.349805:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350422272 00000100:00100000:1.0:1789302824.349827:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350422272 00000100:00100000:1.0:1789302824.349916:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1213 00000100:00100000:1.0:1789302824.349929:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30105:x1876219350422272:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302824.350307:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302824.350341:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e8ac lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0962 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.350468:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.350490:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.351301:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.351354:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.351366:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.351373:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.352773:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e8c1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.352791:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e8c1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.352804:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e8c1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302824.352818:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:1.0:1789302824.354424:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354445:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354455:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354462:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302824.354468:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354495:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354545:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:1.0:1789302824.354549:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302824.354556:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354562:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184e8b3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354581:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354588:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354595:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354600:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302824.354603:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354610:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354617:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f2c94d000) 00010000:00010000:1.0:1789302824.354620:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302824.354623:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.354640:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e8ba lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.356220:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184e8ac lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0962 expref: 111 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.356259:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e8c1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0962 expref: 110 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.356269:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:1.0:1789302824.356273:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302824.356277:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6a00, rc 0) 00000100:00100000:1.0:1789302824.356394:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:30105:x1876219350422272:12345-192.168.203.11@tcp:101:cat.0 Request processed in 6466us (7521us total) trans 4294968100 rc 0/0 00000100:00100000:1.0:1789302824.356405:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1213 00000100:00100000:2.0:1789302824.363065:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.363153:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350422528 00000100:00100000:0.0:1789302824.363167:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350422528 00000100:00100000:0.0:1789302824.363253:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 55 00000100:00100000:0.0:1789302824.363267:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09753050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:30105:x1876219350422528:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:0.0:1789302824.363295:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302824.363330:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184e8c1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0962 expref: 110 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.363354:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:0.0:1789302824.363369:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302824.363443:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09753050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:30105:x1876219350422528:12345-192.168.203.11@tcp:103:cat.0 Request processed in 179us (392us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302824.363451:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 55 00000100:00100000:0.0:1789302824.368039:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.368294:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350422656 00000100:00100000:2.0:1789302824.368309:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350422656 00000100:00100000:2.0:1789302824.368546:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1214 00000100:00100000:2.0:1789302824.368561:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:30105:x1876219350422656:12345-192.168.203.11@tcp:58:cat.0 00010000:00010000:2.0:1789302824.368944:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.369032:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.369050:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789302824.369337:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].35, 136 off12952 00000100:00100000:2.0:1789302824.369638:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:30105:x1876219350422656:12345-192.168.203.11@tcp:58:cat.0 Request processed in 1078us (1673us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.369649:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1214 00000100:00100000:2.0:1789302824.372867:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.373078:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350422784 00000100:00100000:3.0:1789302824.373154:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350422784 00000100:00100000:3.0:1789302824.373273:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1215 00000100:00100000:3.0:1789302824.373286:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:30105:x1876219350422784:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302824.373321:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302824.373351:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0969 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.373433:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.373447:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 110 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.373471:0:18427:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.373486:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 110 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.373651:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302824.373655:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302824.373661:0:18427:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 110 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.373671:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302824.373746:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:30105:x1876219350422784:12345-192.168.203.11@tcp:101:cat.0 Request processed in 463us (898us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.373753:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1215 00000040:00080000:1.0:1789302824.525543:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302824.525556:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302824.525605:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 36 count 4 00000040:00080000:1.0:1789302824.525614:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302824.525781:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70b480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371147392:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302824.527112:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f096f1180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302824.531291:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70b480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371147392:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302824.534498:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302824.534935:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350422912 00000100:00100000:1.0:1789302824.534950:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350422912 00000100:00100000:1.0:1789302824.535041:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1216 00000100:00100000:1.0:1789302824.535053:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350422912:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302824.535126:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302824.535291:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350422912:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 239us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302824.535299:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1216 00000100:00100000:2.0:1789302824.542004:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.542844:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350423040 00000100:00100000:0.0:1789302824.542861:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350423040 00000100:00100000:0.0:1789302824.543045:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1217 00000100:00100000:0.0:1789302824.543058:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350423040:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1789302824.543168:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543195:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543212:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543233:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543241:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543249:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543283:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543291:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543297:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302824.543353:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:0.0:1789302824.543358:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302824.543375:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543389:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543396:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543403:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543408:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302824.543412:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543419:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543430:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302824.543434:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302824.543439:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543443:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e8dd lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543464:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543468:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543472:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543475:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302824.543476:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543480:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543483:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x92f5931a (ffff960f09ae1200) 00010000:00010000:0.0:1789302824.543485:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302824.543486:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543495:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e8e4 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543543:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543549:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543553:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543555:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302824.543557:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543560:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543565:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302824.543567:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302824.543569:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.543573:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e8eb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.543666:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350423040:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 611us (1679us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302824.543672:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1217 00000100:00100000:2.0:1789302824.547227:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.547347:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350423296 00000100:00100000:3.0:1789302824.547372:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350423296 00000100:00100000:3.0:1789302824.547454:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1218 00000100:00100000:3.0:1789302824.547467:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350423296:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302824.547499:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302824.547565:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e8f2 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0970 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547709:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547731:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547751:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302824.547767:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:3.0:1789302824.547910:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547919:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547928:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547934:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302824.547940:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547949:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547965:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:3.0:1789302824.547977:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302824.547984:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.547996:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e8f9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.548262:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e8f2 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0970 expref: 50 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.548277:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302824.548280:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302824.548285:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e8f2 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0970 expref: 50 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.548297:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302824.548473:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:30106:x1876219350423296:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1007us (1260us total) trans 4294968101 rc 301/301 00000100:00100000:3.0:1789302824.548481:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1218 00000100:00100000:3.0:1789302824.552917:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.553014:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350423424 00000100:00100000:0.0:1789302824.553027:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350423424 00000100:00100000:0.0:1789302824.553112:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1219 00000100:00100000:0.0:1789302824.553124:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:30106:x1876219350423424:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302824.553176:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302824.553327:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:30106:x1876219350423424:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 204us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302824.553335:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1219 00000100:00100000:2.0:1789302824.560647:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.560822:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350423680 00000100:00100000:0.0:1789302824.560837:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350423680 00000100:00100000:0.0:1789302824.560923:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1220 00000100:00100000:0.0:1789302824.560936:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:30106:x1876219350423680:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302824.560965:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302824.560994:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e907 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0977 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.562154:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e90e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.562177:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e90e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.562191:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e90e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302824.562204:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x24:0x0]: returning size 0 00010000:00010000:0.0:1789302824.562265:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e907 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0977 expref: 52 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.562288:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e90e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0977 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.562298:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f09ae1200) 00010000:00010000:0.0:1789302824.562301:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302824.562306:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044400, rc 0) 00000100:00100000:0.0:1789302824.565221:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:30106:x1876219350423680:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4267us (4569us total) trans 4294968102 rc 0/0 00000100:00100000:0.0:1789302824.565242:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1220 00000100:00100000:3.0:1789302824.572905:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.573028:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350423936 00000100:00100000:2.0:1789302824.573040:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350423936 00000100:00100000:2.0:1789302824.573118:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1221 00000100:00100000:2.0:1789302824.573130:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:30106:x1876219350423936:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302824.573213:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302824.573247:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e90e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0977 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.573281:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f09ae1200) 00010000:00010000:2.0:1789302824.573294:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302824.573327:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.573346:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 109 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.573356:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 4/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.573415:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 109 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.573430:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 109 pid: 5845 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.573448:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371147520:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302824.573646:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302824.576572:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267fb80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371147520:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302824.576640:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.576972:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302824.577495:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350424064 00000100:00100000:1.0:1789302824.577534:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350424064 00010000:00010000:1.0:1789302824.577583:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 110 pid: 5845 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.577625:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 364 00000100:00100000:1.0:1789302824.577636:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2347:x1876219350424064:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302824.577655:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302824.577663:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 110 pid: 5845 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.577678:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 110 pid: 5845 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.577706:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e820 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a092a expref: 110 pid: 5845 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.577729:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:1.0:1789302824.577736:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.577749:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.577774:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.577811:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302824.577904:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:2347:x1876219350424064:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 267us (933us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302824.577912:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 364 00010000:00010000:2.0:1789302824.578055:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578066:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578100:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578113:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578126:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578413:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578422:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578429:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578436:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302824.578441:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578449:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578467:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:2.0:1789302824.578470:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302824.578486:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578493:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e91c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578536:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578544:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578550:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578556:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302824.578559:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578565:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578574:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x34cf2ce (ffff960f11d36300) 00010000:00010000:2.0:1789302824.578576:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302824.578579:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.578584:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e923 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x34cf2ce bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.578680:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:30106:x1876219350423936:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 5551us (6051us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302824.578689:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1221 00000100:00100000:3.0:1789302824.583242:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.583356:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350424192 00000100:00100000:2.0:1789302824.583378:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350424192 00000100:00100000:2.0:1789302824.583451:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1222 00000100:00100000:2.0:1789302824.583464:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:30106:x1876219350424192:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302824.583492:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302824.583579:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e92a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a097e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.583629:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.583657:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.583672:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.583810:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e92a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 52 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.583831:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.583841:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302824.583853:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302824.583858:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3200, rc 0) 00000100:00100000:2.0:1789302824.583962:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:30106:x1876219350424192:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 493us (727us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.583970:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1222 00010000:00010000:2.0:1789302824.588707:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302824.593454:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.593550:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350424448 00000100:00100000:2.0:1789302824.593566:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350424448 00000100:00100000:2.0:1789302824.593642:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1223 00000100:00100000:2.0:1789302824.593655:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:30106:x1876219350424448:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302824.593768:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.593803:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.593819:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302824.594094:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000024390deb reply state 000000000a7b9d83 transno 4294968103 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.594105:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302824.594217:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:30106:x1876219350424448:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 564us (779us total) trans 4294968103 rc 0/0 00000100:00100000:2.0:1789302824.594225:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1223 00000100:00100000:0.0:1789302824.600477:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302824.600618:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.600633:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.600642:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.600649:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302824.600655:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.600663:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.600686:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302824.600689:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302824.600695:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.600702:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184e938 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.601102:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350424576 00000100:00100000:2.0:1789302824.601116:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350424576 00000100:00100000:2.0:1789302824.601178:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1224 00000100:00100000:2.0:1789302824.601190:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:30106:x1876219350424576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302824.601229:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302824.601252:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 2/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0985 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.601334:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.601351:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.601361:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302824.601363:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302824.601366:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302824.601456:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:30106:x1876219350424576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 268us (995us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.601464:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1224 00000100:00100000:3.0:1789302824.604271:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302824.604369:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350424704 00000100:00100000:1.0:1789302824.604385:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350424704 00000100:00100000:1.0:1789302824.604455:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 244 00000100:00100000:1.0:1789302824.604467:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:30106:x1876219350424704:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1789302824.604895:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:30106:x1876219350424704:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 420us (629us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302824.604904:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 244 00000100:00100000:1.0:1789302824.607106:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.607213:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350424960 00000100:00100000:2.0:1789302824.607226:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350424960 00000100:00100000:2.0:1789302824.607293:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 245 00000100:00100000:2.0:1789302824.607304:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e9850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:30106:x1876219350424960:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302824.607655:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e9850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30106:x1876219350424960:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 352us (561us total) trans 4294968104 rc 0/0 00000100:00100000:2.0:1789302824.607663:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 245 00000100:00100000:0.0:1789302824.612751:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.612866:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350425216 00000100:00100000:2.0:1789302824.612880:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350425216 00000100:00100000:2.0:1789302824.612978:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1225 00000100:00100000:2.0:1789302824.612991:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30106:x1876219350425216:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302824.613166:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30106:x1876219350425216:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 177us (432us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.613175:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1225 00000100:00100000:0.0:1789302824.618548:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.619232:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350425472 00000100:00100000:3.0:1789302824.619249:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350425472 00000100:00100000:3.0:1789302824.619327:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1226 00000100:00100000:3.0:1789302824.619340:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30106:x1876219350425472:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302824.619369:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302824.619400:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e946 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a098c expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.619570:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.619589:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 55 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.619598:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 4/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.619641:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 55 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.619656:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 55 pid: 18427 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.619672:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371147648:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302824.619996:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ee40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302824.623122:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371147648:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302824.623163:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.626393:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.626891:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350425600 00000100:00100000:3.0:1789302824.626905:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350425600 00010000:00010000:3.0:1789302824.626970:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 56 pid: 18427 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.627070:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 365 00000100:00100000:3.0:1789302824.627083:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2347:x1876219350425600:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302824.627104:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302824.627113:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 56 pid: 18427 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.627130:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 56 pid: 18427 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.627158:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184e93f lrc: 0/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0985 expref: 56 pid: 18427 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.627179:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:3.0:1789302824.627186:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.627198:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.627207:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.627249:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302824.627329:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2347:x1876219350425600:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 248us (951us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.627337:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 365 00010000:00010000:0.0:1789302824.627405:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.627416:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302824.627442:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x24:0x0]: returning size 0 00000004:00080000:0.0:1789302824.627921:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x19:0x0], index 1: 0 00010000:00010000:0.0:1789302824.628114:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628124:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628132:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628138:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302824.628143:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628153:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628167:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302824.628170:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302824.628175:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628181:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e94d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628253:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e946 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a098c expref: 55 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628264:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302824.628267:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302824.628271:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184e946 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a098c expref: 55 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.628280:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:0.0:1789302824.628331:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30106:x1876219350425472:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 8995us (9801us total) trans 4294968105 rc 301/301 00000100:00100000:0.0:1789302824.628341:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1226 00000100:00100000:0.0:1789302824.642899:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.644868:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350425856 00000100:00100000:2.0:1789302824.644901:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350425856 00000100:00100000:2.0:1789302824.645000:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 246 00000100:00100000:2.0:1789302824.645013:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30106:x1876219350425856:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302824.645489:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:30106:x1876219350425856:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 477us (2615us total) trans 4294968106 rc 0/0 00000100:00100000:2.0:1789302824.645499:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 246 00000100:00100000:1.0:1789302824.651590:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.651793:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350425984 00000100:00100000:0.0:1789302824.651805:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350425984 00000100:00100000:0.0:1789302824.651871:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1227 00000100:00100000:0.0:1789302824.651891:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:30106:x1876219350425984:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302824.651925:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302824.651964:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e95b lrc: 2/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0993 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.652063:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e962 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.652081:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e962 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.652095:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e962 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302824.652145:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x24:0x0]: returning size 0 00000004:00200000:0.0:1789302824.652151:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302824.652174:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e962 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.652205:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e95b lrc: 0/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0993 expref: 57 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.652227:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e962 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0993 expref: 56 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.652236:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302824.652240:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1789302824.652244:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044400, rc 0) 00000100:00100000:0.0:1789302824.652328:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:30106:x1876219350425984:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 448us (752us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302824.652336:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1227 00000100:00100000:0.0:1789302824.661463:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.663420:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350426112 00000100:00100000:2.0:1789302824.663439:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350426112 00000100:00100000:3.0:1789302824.664096:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 333 00000100:00100000:3.0:1789302824.664113:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350426112:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302824.664141:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302824.664172:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a099a expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302824.664210:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x19:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302824.664238:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=661s v=(0 0) 00010000:00010000:3.0:1789302824.664243:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302824.664256:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.664278:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 10 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.664297:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:3.0:1789302824.664392:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350426112:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 284us (2944us total) trans 0 rc 0/0 00000100:00080000:0.0:1789302824.666820:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302824.667301:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 333 00000100:00100000:1.0:1789302824.682231:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.682354:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350426368 00000100:00100000:2.0:1789302824.682367:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350426368 00000100:00100000:2.0:1789302824.682443:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1228 00000100:00100000:2.0:1789302824.682454:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:30106:x1876219350426368:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302824.683230:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:30106:x1876219350426368:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 771us (1009us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.683241:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1228 00000100:00100000:1.0:1789302824.697209:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.697314:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350426496 00000100:00100000:3.0:1789302824.697327:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350426496 00000100:00100000:3.0:1789302824.697404:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 334 00000100:00100000:3.0:1789302824.697416:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:30106:x1876219350426496:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302824.697442:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302824.697470:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09a1 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.697573:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.697591:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302824.697596:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302824.697602:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.697647:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.697663:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 11 pid: 6696 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.697680:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371147776:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302824.697802:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ee40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302824.705081:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371147776:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302824.705137:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.705164:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302824.705252:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:30106:x1876219350426496:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 7837us (8057us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.705263:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 334 00000100:00100000:0.0:1789302824.709450:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.709584:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350426624 00000100:00100000:2.0:1789302824.709599:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350426624 00010000:00010000:2.0:1789302824.709653:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 11 pid: 6696 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.709707:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:2.0:1789302824.709719:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350426624:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302824.709744:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302824.709776:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 11 pid: 6696 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.709793:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 11 pid: 6696 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.709823:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184e969 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a099a expref: 11 pid: 6696 timeout: 761 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.709845:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302824.709850:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302824.709860:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.709872:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.709935:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.709965:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371147904:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302824.710080:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f4e6cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302824.717896:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267fb80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371147904:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302824.717935:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302824.718028:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350426624:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 8308us (8592us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.718043:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:0.0:1789302824.721208:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.731718:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350426752 00000100:00100000:2.0:1789302824.731739:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350426752 00002000:00100000:2.0:1789302824.731821:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x19:0x0] (0->157) req@ffff960f3f025180 x1876219350426752/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:400/0 lens 648/0 e 0 to 0 dl 1789302835 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302824.731836:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f025180 with x1876219350426752 ext(0->157) 00002000:00010000:2.0:1789302824.731843:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f025180 00000100:00100000:2.0:1789302824.731897:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 109 00000100:00100000:2.0:1789302824.731907:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350426752:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302824.732642:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350426752:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 731us (11445us total) trans 4294967366 rc 0/0 00000100:00100000:2.0:1789302824.732654:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 109 00002000:00100000:2.0:1789302824.732663:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x19:0x0] (0->157) req@ffff960f3f025180 x1876219350426752/t4294967366(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:400/0 lens 648/448 e 0 to 0 dl 1789302835 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302824.732673:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f025180 with x1876219350426752 ext(0->157) 00002000:00010000:2.0:1789302824.732678:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f025180 00000100:00100000:0.0:1789302824.740257:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.740364:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350426880 00000100:00100000:3.0:1789302824.740378:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350426880 00000100:00100000:3.0:1789302824.740451:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1229 00000100:00100000:3.0:1789302824.740464:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:30106:x1876219350426880:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:0.0:1789302824.857381:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968105 is committed 00000001:00080000:0.0:1789302824.857409:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968096 is committed 00000001:00080000:0.0:1789302824.857432:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968100 is committed 00000004:00080000:0.0:1789302824.857439:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 19 committed 00000001:00080000:0.0:1789302824.857480:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968106 is committed 00000020:00080000:0.0:1789302824.857499:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f095bf180 refcount 2 committed 0 result 0 batchid 25 00000020:00080000:0.0:1789302824.857535:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10f38500 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f095bf300 00000020:00080000:0.0:1789302824.857538:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10f38b00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f09672800 00000020:00080000:0.0:1789302824.857719:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 25 committed batchid 24 00000020:00080000:0.0:1789302824.857960:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 24 00000100:00100000:0.0:1789302824.857985:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371148032:0@lo:6:osp-syn-0-0.0 00000100:00100000:0.0:1789302824.858080:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302824.858185:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371148032 00000100:00100000:0.0:1789302824.858195:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371148032 00000100:00100000:0.0:1789302824.858256:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 335 00000100:00100000:0.0:1789302824.858266:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371148032:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:0.0:1789302824.858321:0:6695:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:25 count 1 00002000:00010000:0.0:1789302824.858369:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x19:0x0] initial LVB size: 158, mtime: 0x6aa69826, atime: 0x0, ctime: 0x6aa69826, blocks: 0x1 00010000:00010000:0.0:1789302824.858380:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.858401:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=661s v=(0 0) 00010000:00010000:0.0:1789302824.858407:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302824.858416:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.858435:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.858448:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.858457:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.858479:0:6695:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302824.859077:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371148032:12345-0@lo:6:osp-syn-0-0.0 Request processed in 808us (994us total) trans 4294967366 rc 0/0 00000100:00100000:0.0:1789302824.859087:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 335 00010000:00010000:0.0:1789302824.859130:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302824.859138:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.859148:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302824.859154:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.859165:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.859187:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.859196:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e25ee27/0x64fe43095184e977 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.859218:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302824.859221:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302824.859222:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789302824.859266:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f025500/2, rc 0, transno 4294967366 00000100:00100000:0.0:1789302824.859274:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371148032:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789302824.861277:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30106:x1876219350426880:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 120802us (121023us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.861292:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1229 00000100:00100000:1.0:1789302824.865377:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.865479:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350427008 00000100:00100000:2.0:1789302824.865496:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350427008 00000100:00100000:2.0:1789302824.865600:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 336 00000100:00100000:2.0:1789302824.865612:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f08599450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350427008:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1789302824.915734:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f08599450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350427008:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 50106us (50355us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.915752:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 336 00000001:00080000:3.0:1789302824.916161:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967366 is committed 00000100:00100000:0.0:1789302824.921214:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.921326:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350427264 00000100:00100000:2.0:1789302824.921339:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350427264 00000100:00100000:2.0:1789302824.921421:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1230 00000100:00100000:2.0:1789302824.921432:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30106:x1876219350427264:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302824.921554:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302824.921589:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e962 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0993 expref: 47 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.921614:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302824.921620:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302824.921662:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.921680:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.921692:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302824.921959:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000c29e37ff reply state 000000002d4b1f84 transno 4294968107 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.921972:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302824.922116:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350427264:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 685us (916us total) trans 4294968107 rc 0/0 00000100:00100000:2.0:1789302824.922125:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1230 00000100:00100000:1.0:1789302824.925101:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302824.925233:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.925247:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.925256:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.925262:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302824.925267:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.925276:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.925296:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302824.925299:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302824.925305:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.925311:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184e97e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.925579:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350427392 00000100:00100000:3.0:1789302824.925595:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350427392 00000100:00100000:3.0:1789302824.925677:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1231 00000100:00100000:3.0:1789302824.925692:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30106:x1876219350427392:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302824.925725:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302824.925751:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09a8 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.925827:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.925843:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.925866:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:3.0:1789302824.925869:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302824.925872:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302824.925966:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350427392:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 278us (880us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302824.925974:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1231 00000100:00100000:3.0:1789302824.932190:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302824.932321:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350427520 00000100:00100000:1.0:1789302824.932336:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350427520 00000100:00100000:1.0:1789302824.932415:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:1.0:1789302824.932428:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350427520:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:1.0:1789302824.932821:0:8571:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:1.0:1789302824.932827:0:8571:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb mtime from disk: 0 -> 1789302822 00002000:00010000:1.0:1789302824.932830:0:8571:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb atime from disk: 0 -> 1789302823 00002000:00010000:1.0:1789302824.932832:0:8571:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb ctime from disk: 0 -> 1789302822 00002000:00010000:1.0:1789302824.932834:0:8571:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x19:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:1.0:1789302824.932922:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350427520:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 496us (747us total) trans 4294967367 rc 0/0 00000100:00100000:1.0:1789302824.932932:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:2.0:1789302824.941176:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302824.941288:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350427776 00000100:00100000:3.0:1789302824.941302:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350427776 00000100:00100000:3.0:1789302824.941390:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1232 00000100:00100000:3.0:1789302824.941402:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30106:x1876219350427776:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1789302824.941536:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.941558:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.941569:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 4/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.941613:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.941630:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 48 pid: 5844 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.941647:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371148160:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302824.941781:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4ed80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302824.951991:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302824.952109:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350427904 00000100:00100000:1.0:1789302824.952123:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350427904 00010000:00010000:1.0:1789302824.952188:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 49 pid: 5844 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.952241:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 367 00000100:00100000:1.0:1789302824.952254:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350427904:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302824.952274:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302824.952282:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 49 pid: 5844 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.952298:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 49 pid: 5844 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.952327:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302824.952339:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.952351:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.952361:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302824.952387:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302824.952467:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350427904:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 215us (489us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302824.952476:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 367 00010000:00010000:3.0:1789302824.952940:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3200/0x64fe43095184e931 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x24:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a097e expref: 48 pid: 5844 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.952963:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371148160:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302824.952992:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953007:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953014:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953055:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953062:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953070:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953076:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302824.953082:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953091:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953103:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:3.0:1789302824.953106:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:3.0:1789302824.953111:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 2/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302824.953117:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e98c lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302824.955158:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30106:x1876219350427776:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 13742us (13979us total) trans 0 rc -22/-22 00000100:00100000:3.0:1789302824.955178:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1232 00000100:00100000:1.0:1789302824.971337:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302824.971486:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350428032 00000100:00100000:0.0:1789302824.971669:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350428032 00000100:00100000:0.0:1789302824.971751:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:0.0:1789302824.971765:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350428032:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302824.971830:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.971856:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.971866:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302824.971872:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302824.971878:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.971943:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.971961:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 11 pid: 18578 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.971981:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371148288:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302824.972271:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4d80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302824.983914:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371148288:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302824.983963:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302824.986389:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302824.988252:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350428160 00000100:00100000:2.0:1789302824.988274:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350428160 00010000:00010000:2.0:1789302824.988342:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 12 pid: 18578 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302824.988402:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 368 00000100:00100000:2.0:1789302824.988413:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350428160:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302824.988433:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302824.988464:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 12 pid: 18578 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.988482:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 12 pid: 18578 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.988642:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184e970 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a1 expref: 12 pid: 18578 timeout: 761 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.988668:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302824.988673:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302824.988680:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.988692:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302824.988734:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302824.988820:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350428160:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 408us (2445us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302824.988830:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 368 00010000:00010000:0.0:1789302824.989100:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989112:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989157:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989168:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989183:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989192:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302824.989198:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989208:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989223:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302824.989232:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000033cb9bc6/0x64fe43095184e993 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302824.989313:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350428032:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 17549us (17993us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302824.989323:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:1.0:1789302825.005484:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302825.009708:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350428416 00000100:00100000:0.0:1789302825.009736:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350428416 00000100:00100000:0.0:1789302825.009819:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1233 00000100:00100000:0.0:1789302825.009836:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30106:x1876219350428416:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302825.009991:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.010016:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.010034:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.015225:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.015260:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.015271:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.015330:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.015351:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 48 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302825.015368:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371148416:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302825.015488:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4d80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302825.028091:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371148416:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302825.028141:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302825.030441:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302825.030566:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350428544 00000100:00100000:2.0:1789302825.030580:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350428544 00010000:00010000:2.0:1789302825.030634:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 49 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302825.030683:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 369 00000100:00100000:2.0:1789302825.030697:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350428544:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302825.030721:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302825.030730:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 49 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.030746:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 49 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.030774:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184e985 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09a8 expref: 49 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.030793:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302825.030800:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.030810:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.030818:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.030856:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302825.030926:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.030937:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302825.030944:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2347:x1876219350428544:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 249us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.030962:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 369 00000001:00080000:0.0:1789302825.031209:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968108 00000001:00080000:0.0:1789302825.031665:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968109 00000001:00080000:0.0:1789302825.031718:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968110 00000100:00100000:0.0:1789302825.031818:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371148544:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1789302825.031898:0:5843:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302825.031936:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302825.031954:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371148544 00000100:00100000:2.0:1789302825.031964:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371148544 00000100:00100000:2.0:1789302825.032030:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 55 00000100:00100000:2.0:1789302825.032042:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371148544:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1789302825.032151:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371148544:12345-0@lo:1000:mdt00_000.0 Request processed in 112us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.032160:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 55 00000100:00100000:1.0:1789302825.032243:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371148544:0@lo:1000:mdt00_000.0 00000001:00080000:1.0:1789302825.033017:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968111 00000100:00100000:1.0:1789302825.033099:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371148672:0@lo:1000:mdt00_000.0 00000100:00100000:1.0:1789302825.033201:0:5843:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302825.033238:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302825.033258:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371148672 00000100:00100000:2.0:1789302825.033267:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371148672 00000100:00100000:2.0:1789302825.033318:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 56 00000100:00100000:2.0:1789302825.033327:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371148672:12345-0@lo:1000:mdt00_000.0 00000100:00080000:0.0:1789302825.033486:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302825.033761:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5843:x1876219371148672:12345-0@lo:1000:mdt00_000.0 Request processed in 434us (558us total) trans 4294967368 rc 0/0 00000100:00100000:2.0:1789302825.033769:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 56 00000100:00100000:1.0:1789302825.033933:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371148672:0@lo:1000:mdt00_000.0 00000001:00080000:1.0:1789302825.034029:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968112 00010000:00010000:1.0:1789302825.034065:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034076:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034085:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034092:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302825.034098:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034107:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034153:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302825.034157:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302825.034165:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034172:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184e9a1 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x24:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034209:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034215:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034221:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034227:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302825.034230:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034236:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034244:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302825.034247:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302825.034255:0:5843:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034270:0:5843:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034286:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.034321:0:5843:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302825.034334:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095adf80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371148800:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302825.034413:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b33c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302825.037351:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302825.037458:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350428672 00000100:00100000:3.0:1789302825.037471:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350428672 00000100:00100000:1.0:1789302825.037609:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095adf80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371148800:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302825.037623:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 247 00000100:00100000:3.0:1789302825.037635:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30105:x1876219350428672:12345-192.168.203.11@tcp:49:cat.0 00010000:00010000:1.0:1789302825.037660:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037674:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184e8cf lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037714:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037733:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037748:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037759:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037765:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037781:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037787:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00000100:00100000:3.0:1789302825.037790:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30105:x1876219350428672:12345-192.168.203.11@tcp:49:cat.0 Request processed in 156us (454us total) trans 0 rc 0/0 00010000:00010000:1.0:1789302825.037793:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302825.037805:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 247 00010000:00010000:1.0:1789302825.037807:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037824:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302825.037828:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302825.037833:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037840:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095184e9a8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037851:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037858:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037864:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037870:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302825.037874:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037903:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037912:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302825.037915:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302825.037919:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 2/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.037925:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e99a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302825.038821:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:30106:x1876219350428416:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 28985us (33341us total) trans 4294968113 rc 0/0 00000100:00100000:1.0:1789302825.038834:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1233 00000020:00080000:1.0:1789302825.038840:0:5843:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000100:00100000:2.0:1789302825.046404:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302825.046455:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302825.046849:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350428928 00000100:00100000:3.0:1789302825.046863:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350428928 00000100:00100000:3.0:1789302825.046940:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 248 00000100:00100000:3.0:1789302825.046951:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:30106:x1876219350428928:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1789302825.048003:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x24:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1789302825.048118:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968114 00010000:00010000:3.0:1789302825.048611:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 3/0,1 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.048634:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.048649:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.048659:0:5846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.048667:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.048680:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.048969:0:5846:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302825.049115:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30106:x1876219350428928:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 2161us (2721us total) trans 4294968115 rc 0/0 00010000:00010000:1.0:1789302825.049199:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302825.049208:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.049217:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302825.049221:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.049233:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.049248:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x24:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302825.049251:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789302825.049257:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 1/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.049276:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184e9af lrc: 0/0,0 mode: --/PW res: [0x200000402:0x24:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.049296:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302825.049298:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302825.049299:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302825.049365:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302825.049371:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:3.0:1789302825.049680:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 248 00000020:00080000:3.0:1789302825.049690:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789302825.049733:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350429056 00000100:00100000:3.0:1789302825.049744:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350429056 00000100:00080000:2.0:1789302825.049801:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302825.049812:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:3.0:1789302825.049822:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:30105:x1876219350429056:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302825.049840:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302825.049860:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09af expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302825.049896:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x19:0x0] initial LVB size: 158, mtime: 0x6aa69826, atime: 0x6aa69827, ctime: 0x6aa69826, blocks: 0xb 00010000:00010000:3.0:1789302825.049921:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=662s v=(0 0) 00010000:00010000:3.0:1789302825.049926:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302825.049937:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 18 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.049955:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 18 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.049966:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302825.051332:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:30105:x1876219350429056:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1498us (4863us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302825.051346:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 339 00000040:00080000:1.0:1789302825.058637:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:1.0:1789302825.058668:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302825.064844:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302825.065169:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350429184 00000100:00100000:3.0:1789302825.065184:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350429184 00002000:00100000:3.0:1789302825.065250:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x19:0x0] (0->4095) req@ffff960e0267f800 x1876219350429184/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:401/0 lens 488/0 e 0 to 0 dl 1789302836 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302825.065265:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0267f800 with x1876219350429184 ext(0->4095) 00002000:00010000:3.0:1789302825.065272:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0267f800 00000100:00100000:3.0:1789302825.065301:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 110 00000100:00100000:3.0:1789302825.065313:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350429184:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:3.0:1789302825.065621:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350429184:12345-192.168.203.11@tcp:3:cat.0 Request processed in 307us (791us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302825.065636:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 110 00002000:00100000:3.0:1789302825.065645:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x19:0x0] (0->4095) req@ffff960e0267f800 x1876219350429184/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:401/0 lens 488/4536 e 0 to 0 dl 1789302836 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302825.065655:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0267f800 with x1876219350429184 ext(0->4095) 00002000:00010000:3.0:1789302825.065659:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0267f800 00000001:00080000:2.0:1789302825.066532:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967366 is committed 00000001:00080000:2.0:1789302825.066559:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967364 is committed 00000100:00100000:2.0:1789302825.078532:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302825.078593:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350429440 00000100:00100000:1.0:1789302825.078609:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350429440 00000100:00100000:1.0:1789302825.078677:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:1.0:1789302825.078690:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30105:x1876219350429440:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302825.079072:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30105:x1876219350429440:12345-192.168.203.11@tcp:35:cat.0 Request processed in 384us (572us total) trans 4294968116 rc 0/0 00000100:00100000:1.0:1789302825.079081:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 249 00000100:00100000:2.0:1789302825.092442:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302825.092557:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350429568 00000100:00100000:0.0:1789302825.092572:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350429568 00000100:00100000:0.0:1789302825.092651:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1234 00000100:00100000:0.0:1789302825.092663:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30107:x1876219350429568:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302825.092698:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302825.092732:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e9bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09b6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.092833:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.092853:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.092869:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.092900:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.092907:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.092919:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093226:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093237:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093245:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302825.093409:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:0.0:1789302825.093415:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302825.093442:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093458:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093472:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093479:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093486:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093493:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302825.093498:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093653:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093845:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:0.0:1789302825.093849:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302825.093856:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093864:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184e9c4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093900:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093907:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093913:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093918:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302825.093922:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093928:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093935:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f09ae1800) 00010000:00010000:0.0:1789302825.093937:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302825.093940:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.093950:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.094008:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e9bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 100 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.094024:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.094031:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302825.094034:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302825.094036:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045000, rc 0) 00000100:00100000:2.0:1789302825.095602:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30107:x1876219350429568:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2933us (3168us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.095615:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1234 00000100:00100000:2.0:1789302825.101682:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302825.102082:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350429696 00000100:00100000:3.0:1789302825.102098:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350429696 00000100:00100000:3.0:1789302825.102616:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1235 00000100:00100000:3.0:1789302825.102633:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30107:x1876219350429696:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302825.102662:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302825.102690:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e9d9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09bd expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.102796:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.102814:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.102828:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302825.102873:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x48:0x0]: returning size 16384 00000004:00200000:3.0:1789302825.102889:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302825.102912:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.102944:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184e9d9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 101 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.102977:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 100 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.102986:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:3.0:1789302825.102990:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302825.102994:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302825.103071:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30107:x1876219350429696:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 445us (1404us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302825.103079:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1235 00000100:00100000:1.0:1789302825.110684:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302825.110788:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350429952 00000100:00100000:0.0:1789302825.110798:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350429952 00000100:00100000:0.0:1789302825.110862:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1236 00000100:00100000:0.0:1789302825.110873:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30107:x1876219350429952:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302825.110904:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302825.110925:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e9e7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09c4 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.111021:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9ee lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.111036:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.111048:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9ee lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302825.111065:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:0.0:1789302825.111180:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184e9e7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09c4 expref: 103 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.111200:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9ee lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09c4 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.111208:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302825.111212:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302825.111216:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045c00, rc 0) 00000100:00100000:0.0:1789302825.111275:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30107:x1876219350429952:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 404us (599us total) trans 4294968117 rc 0/0 00000100:00100000:0.0:1789302825.111286:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1236 00000100:00100000:1.0:1789302825.116787:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302825.116910:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350430080 00000100:00100000:2.0:1789302825.116921:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350430080 00000100:00100000:2.0:1789302825.116989:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:2.0:1789302825.117001:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350430080:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:2.0:1789302825.117047:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,1 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.117070:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 18 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.117079:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302825.117083:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302825.117087:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 4/0,1 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.117124:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 18 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.117139:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 18 pid: 6696 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302825.117154:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371148928:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302825.117489:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302825.131577:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302825.131654:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350430208 00000100:00100000:3.0:1789302825.131663:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350430208 00010000:00010000:3.0:1789302825.131779:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 19 pid: 6696 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302825.131818:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 370 00000100:00100000:3.0:1789302825.131827:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09753850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2348:x1876219350430208:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302825.131842:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302825.131865:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 19 pid: 6696 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.131887:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 19 pid: 6696 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.131911:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302825.131916:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302825.131922:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.131933:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.131955:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302825.132010:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09753850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2348:x1876219350430208:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 184us (441us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302825.132018:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 370 00010000:00010000:2.0:1789302825.133629:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f8011d48/0x64fe43095184e9b6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x19:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09af expref: 18 pid: 6696 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302825.133650:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371148928:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302825.133671:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133683:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133690:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133722:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133731:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133738:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133744:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302825.133748:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133756:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133772:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 2/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.133779:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000810a9f4e/0x64fe43095184e9fc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302825.133923:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350430080:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 16925us (17145us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.133932:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:1.0:1789302825.142923:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302825.143009:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350430464 00000100:00100000:3.0:1789302825.143018:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350430464 00000100:00100000:3.0:1789302825.143079:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 250 00000100:00100000:3.0:1789302825.143090:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e8450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30107:x1876219350430464:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:3.0:1789302825.143230:0:5846:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x23:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9ee lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09c4 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302825.144113:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968118 00000001:00080000:3.0:1789302825.144596:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968119 00010000:00010000:3.0:1789302825.144658:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.144676:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.144685:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 105 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.144693:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 5/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.144731:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 105 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.144749:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 105 pid: 5845 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302825.144766:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371149056:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302825.144866:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302825.144891:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 105 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302825.144900:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371149184:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302825.144924:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302825.148733:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371149056:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302825.156267:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302825.156343:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371149184:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302825.156370:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350430592 00000100:00100000:2.0:1789302825.156384:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350430592 00010000:00010000:3.0:1789302825.156394:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.156428:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 106 pid: 5845 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302825.156477:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 371 00000100:00100000:2.0:1789302825.156487:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350430592:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302825.156540:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302825.156548:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 106 pid: 5845 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.156562:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 106 pid: 5845 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.156584:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184e9d2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09b6 expref: 106 pid: 5845 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.156604:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:2.0:1789302825.156610:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.156619:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.156626:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302825.159774:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302825.160359:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350430720 00000100:00100000:0.0:1789302825.160369:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350430720 00010000:00010000:0.0:1789302825.160410:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 106 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302825.160451:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 372 00000100:00100000:0.0:1789302825.160462:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350430720:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302825.160477:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302825.160484:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 106 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.160498:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 106 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.160566:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184e8d6 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0969 expref: 106 pid: 18427 timeout: 762 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.160582:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:0.0:1789302825.160587:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.160597:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.160604:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302825.160633:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302825.160679:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302825.160694:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:2347:x1876219350430720:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 228us (997us total) trans 0 rc 0/0 00010000:00010000:3.0:1789302825.160705:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302825.160711:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 372 00000100:00100000:2.0:1789302825.160864:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350430592:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4377us (4603us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.160873:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 371 00000001:00080000:3.0:1789302825.160932:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968120 00000001:00080000:3.0:1789302825.161218:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968121 00000001:00080000:3.0:1789302825.161275:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968122 00000100:00100000:3.0:1789302825.161384:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371149312:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1789302825.161460:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302825.163274:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371149312 00000100:00100000:2.0:1789302825.163294:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371149312 00000100:00100000:2.0:1789302825.163377:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 57 00000100:00100000:2.0:1789302825.163392:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5846:x1876219371149312:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302825.164261:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5846:x1876219371149312:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 869us (2795us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.164273:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 57 00000100:00100000:3.0:1789302825.164860:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:3.0:1789302825.164918:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371149312:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:3.0:1789302825.166096:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968123 00000100:00100000:3.0:1789302825.166233:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371149440:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:3.0:1789302825.166319:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302825.166368:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e6c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302825.166430:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371149440 00000100:00100000:0.0:1789302825.166456:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371149440 00000100:00100000:0.0:1789302825.166551:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 58 00000100:00100000:0.0:1789302825.166562:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5846:x1876219371149440:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:0.0:1789302825.166852:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+7:5846:x1876219371149440:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 291us (531us total) trans 4294967369 rc 0/0 00000100:00100000:0.0:1789302825.166861:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 58 00000100:00100000:1.0:1789302825.166931:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371149440:0@lo:1000:mdt_rdpg00_000.0 00010000:00010000:1.0:1789302825.166977:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.167009:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.167018:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.167027:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302825.167033:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.167042:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.167062:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302825.167065:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:1.0:1789302825.167072:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.167078:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea03 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302825.167334:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x4a:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:1.0:1789302825.167478:0:5846:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].11, 64 off8968 00000004:00080000:1.0:1789302825.167492:0:5846:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 662177989346ns, next at 663177986158ns, rc = 0 00000001:00080000:1.0:1789302825.167528:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968124 00010000:00010000:1.0:1789302825.167594:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f02dfd900) 00010000:00010000:1.0:1789302825.167597:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1789302825.167608:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184e9ee lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09c4 expref: 108 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302825.167764:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e8450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:30107:x1876219350430464:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 24676us (24849us total) trans 4294968125 rc 0/0 00000100:00100000:1.0:1789302825.167774:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 250 00000020:00080000:1.0:1789302825.167780:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:1.0:1789302825.250379:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302825.250496:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350430976 00000100:00100000:2.0:1789302825.250549:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350430976 00000100:00100000:2.0:1789302825.250623:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1237 00000100:00100000:2.0:1789302825.250635:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:30113:x1876219350430976:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302825.250669:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302825.250699:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea0a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09cb expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.250806:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.250827:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.250842:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.250866:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.250874:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.250892:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251278:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ea1f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251288:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ea1f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251296:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ea1f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302825.251307:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:2.0:1789302825.251441:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251450:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251458:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251465:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302825.251471:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251483:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251498:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:2.0:1789302825.251529:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302825.251535:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251542:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ea11 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251555:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251561:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251567:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251572:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302825.251575:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251582:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251589:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0xb3a1d06f (ffff960f03927400) 00010000:00010000:2.0:1789302825.251592:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302825.251595:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251606:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0xb3a1d06f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251660:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea0a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09cb expref: 110 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251677:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ea1f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09cb expref: 109 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.251685:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302825.251687:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302825.251690:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302825.251782:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:30113:x1876219350430976:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1148us (1416us total) trans 4294968126 rc 0/0 00000100:00100000:2.0:1789302825.251790:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1237 00000100:00100000:2.0:1789302825.257488:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302825.258022:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350431232 00000100:00100000:1.0:1789302825.258045:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350431232 00000100:00100000:1.0:1789302825.258161:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 56 00000100:00100000:1.0:1789302825.258174:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:30113:x1876219350431232:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:1.0:1789302825.258194:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302825.258228:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ea1f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09cb expref: 109 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302825.258251:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302825.258265:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302825.258345:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30113:x1876219350431232:12345-192.168.203.11@tcp:103:cat.0 Request processed in 173us (872us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302825.258354:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 56 00000100:00100000:1.0:1789302825.271476:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302825.271705:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350431360 00000100:00100000:2.0:1789302825.271720:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350431360 00000100:00100000:2.0:1789302825.271793:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1238 00000100:00100000:2.0:1789302825.271806:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30113:x1876219350431360:12345-192.168.203.11@tcp:58:cat.0 00010000:00010000:2.0:1789302825.271968:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.271992:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.272007:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789302825.273314:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].36, 136 off13088 00000100:00100000:2.0:1789302825.273493:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30113:x1876219350431360:12345-192.168.203.11@tcp:58:cat.0 Request processed in 1685us (2028us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.273527:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1238 00000100:00100000:3.0:1789302825.278632:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302825.278796:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350431488 00000100:00100000:2.0:1789302825.278810:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350431488 00000100:00100000:2.0:1789302825.278895:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1239 00000100:00100000:2.0:1789302825.278909:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30113:x1876219350431488:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302825.278939:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302825.279222:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09d2 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.279370:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.279384:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d2 expref: 109 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.279407:0:5844:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.279423:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d2 expref: 109 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.279440:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302825.279443:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302825.279447:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d2 expref: 109 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302825.279456:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3800, rc 0) 00000100:00100000:2.0:1789302825.279781:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:30113:x1876219350431488:12345-192.168.203.11@tcp:101:cat.0 Request processed in 872us (1161us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302825.279789:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1239 00010000:00010000:2.0:1789302825.613737:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302826.065153:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302826.065163:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302826.065202:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 37 count 5 00000040:00080000:1.0:1789302826.065209:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302826.065346:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ae680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371149568:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302826.065475:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302826.070182:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ae680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371149568:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302826.076089:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.076280:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350431616 00000100:00100000:2.0:1789302826.076297:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350431616 00000100:00080000:0.0:1789302826.076429:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302826.076444:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1240 00000100:00100000:2.0:1789302826.076464:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30117:x1876219350431616:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302826.076556:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302826.076701:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30117:x1876219350431616:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 243us (2869us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.076709:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1240 00000100:00100000:3.0:1789302826.086601:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.087006:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350431744 00000100:00100000:2.0:1789302826.087021:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350431744 00000100:00100000:2.0:1789302826.087096:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1241 00000100:00100000:2.0:1789302826.087109:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30117:x1876219350431744:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1789302826.087229:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087253:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087271:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087295:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087303:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087309:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087349:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087358:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087365:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302826.087419:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00000004:00200000:2.0:1789302826.087424:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302826.087443:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087455:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087462:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087469:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087480:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302826.087485:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087528:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087546:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302826.087550:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302826.087557:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087564:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ea3b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087578:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087584:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087591:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087596:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302826.087600:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087606:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087618:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x92f5931a (ffff960f03927100) 00010000:00010000:2.0:1789302826.087621:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302826.087625:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087630:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ea42 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x92f5931a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087644:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087650:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087656:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087661:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302826.087664:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087669:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087677:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302826.087679:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302826.087682:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.087688:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea49 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.087807:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30117:x1876219350431744:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 696us (1218us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.087818:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1241 00000100:00100000:3.0:1789302826.090439:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.090767:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350432000 00000100:00100000:2.0:1789302826.090783:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350432000 00000100:00100000:2.0:1789302826.090963:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1242 00000100:00100000:2.0:1789302826.090976:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30117:x1876219350432000:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302826.091006:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302826.091034:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea50 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09d9 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091166:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091185:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091200:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302826.091214:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x23:0x0]: returning size 158 00010000:00010000:2.0:1789302826.091356:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091365:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091373:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091386:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302826.091391:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091399:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091415:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927400) 00010000:00010000:2.0:1789302826.091418:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302826.091424:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091430:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea57 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x23:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091484:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea50 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d9 expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091528:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302826.091532:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302826.091537:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea50 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d9 expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.091549:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:2.0:1789302826.091641:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:30117:x1876219350432000:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 667us (1219us total) trans 4294968127 rc 301/301 00000100:00100000:2.0:1789302826.091649:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1242 00000100:00100000:1.0:1789302826.096549:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.096688:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350432128 00000100:00100000:0.0:1789302826.096701:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350432128 00000100:00100000:0.0:1789302826.096775:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1243 00000100:00100000:0.0:1789302826.096799:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:30117:x1876219350432128:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302826.096856:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x23:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302826.097022:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:30117:x1876219350432128:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 237us (528us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302826.097031:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1243 00000100:00100000:3.0:1789302826.103380:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.106578:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350432384 00000100:00100000:0.0:1789302826.106604:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350432384 00000100:00100000:0.0:1789302826.106710:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1244 00000100:00100000:0.0:1789302826.106724:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:30117:x1876219350432384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302826.106760:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302826.107070:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ea65 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09e0 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.111846:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ea6c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.111885:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ea6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.111903:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ea6c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302826.111918:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x25:0x0]: returning size 0 00010000:00010000:0.0:1789302826.113323:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ea65 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e0 expref: 58 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.113358:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ea6c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e0 expref: 57 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.113368:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f09ae1800) 00010000:00010000:0.0:1789302826.113372:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302826.113376:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045c00, rc 0) 00000100:00100000:0.0:1789302826.113467:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:30117:x1876219350432384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 6746us (10102us total) trans 4294968128 rc 0/0 00000100:00100000:0.0:1789302826.113478:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1244 00000100:00100000:3.0:1789302826.121476:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.121656:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350432640 00000100:00100000:2.0:1789302826.121671:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350432640 00000100:00100000:2.0:1789302826.121757:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1245 00000100:00100000:2.0:1789302826.121770:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:30117:x1876219350432640:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302826.122354:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302826.122393:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ea6c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e0 expref: 57 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.122418:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f09ae1800) 00010000:00010000:2.0:1789302826.122431:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302826.122463:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.122480:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 108 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.122490:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 4/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.122673:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 108 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.122691:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 108 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.122717:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371149696:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302826.122844:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302826.128448:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267c000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371149696:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302826.128491:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.128902:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302826.129313:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350432768 00000100:00100000:3.0:1789302826.129324:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350432768 00010000:00010000:3.0:1789302826.129439:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 109 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302826.129486:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 373 00000100:00100000:3.0:1789302826.129497:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:2347:x1876219350432768:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302826.129560:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302826.129569:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 109 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129583:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 109 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129610:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184e9e0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x48:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09bd expref: 109 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129635:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:3.0:1789302826.129641:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129652:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129661:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129693:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302826.129769:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350432768:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 273us (868us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302826.129778:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 373 00010000:00010000:3.0:1789302826.129808:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129816:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129843:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129853:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.129860:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130449:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130461:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130469:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130476:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302826.130482:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130490:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130528:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:3.0:1789302826.130532:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302826.130538:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130544:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184ea7a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x48:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130557:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130562:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130568:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130573:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302826.130577:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130583:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130590:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x172ef7a (ffff960f0cdf2700) 00010000:00010000:3.0:1789302826.130593:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302826.130597:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.130603:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184ea81 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x48:0x0].0x172ef7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302826.130685:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:30117:x1876219350432640:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 8916us (9221us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302826.130694:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1245 00000100:00080000:0.0:1789302826.131310:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302826.133499:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.133742:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350432896 00000100:00100000:2.0:1789302826.133756:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350432896 00000100:00100000:2.0:1789302826.133881:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1246 00000100:00100000:2.0:1789302826.133894:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:30117:x1876219350432896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302826.133931:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302826.133968:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea88 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09e7 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.134013:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.134029:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.134043:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.134553:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ea88 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 58 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.134577:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 57 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.134586:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302826.134590:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302826.134594:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2200, rc 0) 00000100:00100000:2.0:1789302826.134675:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:30117:x1876219350432896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 783us (1189us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.134683:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1246 00000100:00100000:0.0:1789302826.138717:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302826.139618:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350433152 00000100:00100000:3.0:1789302826.139634:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350433152 00000100:00100000:3.0:1789302826.139704:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1247 00000100:00100000:3.0:1789302826.139717:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:30117:x1876219350433152:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302826.140062:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.140086:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.140101:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302826.140390:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009602db8e reply state 0000000012d3581d transno 4294968129 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302826.140401:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302826.140595:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:30117:x1876219350433152:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 878us (1891us total) trans 4294968129 rc 0/0 00000100:00100000:3.0:1789302826.140603:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1247 00010000:00010000:1.0:1789302826.144011:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.144035:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302826.144045:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302826.144049:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.144057:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302826.144063:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.144073:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.144095:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302826.144099:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302826.144104:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.144111:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ea96 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302826.144935:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350433280 00000100:00100000:0.0:1789302826.145007:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350433280 00000100:00100000:0.0:1789302826.145090:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1248 00000100:00100000:0.0:1789302826.145103:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:30117:x1876219350433280:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302826.145140:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302826.145163:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09ee expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.145238:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 59 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.145256:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 59 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.145265:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302826.145269:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302826.145343:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044400, rc 0) 00000100:00100000:0.0:1789302826.145499:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:30117:x1876219350433280:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 398us (1464us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302826.145587:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1248 00000100:00100000:1.0:1789302826.147908:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.148021:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350433408 00000100:00100000:2.0:1789302826.148034:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350433408 00000100:00100000:2.0:1789302826.148103:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 251 00000100:00100000:2.0:1789302826.148115:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:30117:x1876219350433408:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:2.0:1789302826.148371:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:30117:x1876219350433408:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 258us (476us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.148380:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 251 00000100:00100000:2.0:1789302826.153335:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.153546:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350433664 00000100:00100000:1.0:1789302826.153562:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350433664 00000100:00100000:1.0:1789302826.153653:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 252 00000100:00100000:1.0:1789302826.153665:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:30117:x1876219350433664:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302826.154029:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:30117:x1876219350433664:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 364us (709us total) trans 4294968130 rc 0/0 00000100:00100000:1.0:1789302826.154038:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 252 00000100:00100000:3.0:1789302826.161593:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.161925:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350433920 00000100:00100000:2.0:1789302826.161940:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350433920 00000100:00100000:2.0:1789302826.162024:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1249 00000100:00100000:2.0:1789302826.162037:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:30117:x1876219350433920:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302826.162210:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:30117:x1876219350433920:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 174us (630us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.162219:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1249 00000100:00100000:0.0:1789302826.165778:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.166196:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350434176 00000100:00100000:1.0:1789302826.166212:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350434176 00000100:00100000:1.0:1789302826.166292:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1250 00000100:00100000:1.0:1789302826.166305:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:30117:x1876219350434176:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302826.166338:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302826.166369:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eaa4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09f5 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.167266:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.168137:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 61 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.168149:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 4/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.168200:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 61 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.168221:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 61 pid: 5844 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302826.168265:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371149824:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302826.168415:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3a80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302826.171895:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.172064:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371149824:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302826.172106:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.172922:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350434304 00000100:00100000:2.0:1789302826.172935:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350434304 00010000:00010000:2.0:1789302826.172992:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 62 pid: 5844 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.173169:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 374 00000100:00100000:2.0:1789302826.173181:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:2348:x1876219350434304:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302826.173201:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302826.173210:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 62 pid: 5844 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.173224:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 62 pid: 5844 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.173251:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ea9d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09ee expref: 62 pid: 5844 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.173272:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302826.173278:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.173290:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.173299:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.173335:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302826.173414:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:2348:x1876219350434304:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 235us (1545us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.173424:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 374 00010000:00010000:1.0:1789302826.173491:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.173516:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302826.173542:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x25:0x0]: returning size 0 00000004:00080000:1.0:1789302826.173998:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1a:0x0], index 0: 0 00010000:00010000:1.0:1789302826.174175:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174185:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174192:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174199:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302826.174204:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174213:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174241:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302826.174244:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302826.174250:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 2/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174256:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184eaab lrc: 0/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174320:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eaa4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09f5 expref: 61 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174331:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x48:0x0].0x0 (ffff960f2c94d400) 00010000:00010000:1.0:1789302826.174333:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302826.174337:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eaa4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x48:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09f5 expref: 61 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.174346:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302826.174385:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:30117:x1876219350434176:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 8085us (8625us total) trans 4294968131 rc 301/301 00000100:00100000:1.0:1789302826.174394:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1250 00000100:00100000:3.0:1789302826.182963:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.183110:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350434560 00000100:00100000:2.0:1789302826.183124:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350434560 00000100:00100000:2.0:1789302826.183193:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:2.0:1789302826.183206:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:30117:x1876219350434560:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302826.183959:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:30117:x1876219350434560:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 744us (1010us total) trans 4294968132 rc 0/0 00000100:00100000:2.0:1789302826.183969:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 253 00000100:00100000:3.0:1789302826.187671:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.187805:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350434688 00000100:00100000:0.0:1789302826.187822:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350434688 00000100:00100000:0.0:1789302826.187962:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1251 00000100:00100000:0.0:1789302826.187974:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:30117:x1876219350434688:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302826.188009:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302826.188038:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eab9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a09fc expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.188142:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eac0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.188161:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eac0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.188177:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eac0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302826.188226:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x25:0x0]: returning size 0 00000004:00200000:0.0:1789302826.188231:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302826.188253:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eac0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.188303:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eab9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09fc expref: 63 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.188328:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eac0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09fc expref: 62 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.188341:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302826.188344:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:0.0:1789302826.188350:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045a00, rc 0) 00000100:00100000:0.0:1789302826.188472:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:30117:x1876219350434688:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 500us (819us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302826.188481:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1251 00000100:00100000:3.0:1789302826.191287:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.191684:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350434816 00000100:00100000:1.0:1789302826.191699:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350434816 00000100:00100000:1.0:1789302826.191788:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:1.0:1789302826.191801:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350434816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302826.191827:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302826.191856:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a03 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302826.191919:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789302826.191954:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=663s v=(0 0) 00010000:00010000:1.0:1789302826.191959:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302826.191971:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.191996:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.192012:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302826.192087:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350434816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 289us (816us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302826.192094:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00080000:1.0:1789302826.192629:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302826.195325:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.195493:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350435072 00000100:00100000:0.0:1789302826.195585:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350435072 00000100:00100000:0.0:1789302826.195706:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1252 00000100:00100000:0.0:1789302826.195719:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:30117:x1876219350435072:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302826.195868:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:30117:x1876219350435072:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 151us (556us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302826.195885:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1252 00000100:00100000:3.0:1789302826.198070:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.198180:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350435200 00000100:00100000:1.0:1789302826.198193:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350435200 00000100:00100000:1.0:1789302826.198263:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 342 00000100:00100000:1.0:1789302826.198275:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:30117:x1876219350435200:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302826.198299:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302826.198324:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a0a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.198365:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.198401:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302826.198407:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302826.198413:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.198455:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.198472:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 11 pid: 13439 timeout: 763 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302826.198490:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371149952:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302826.198617:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302826.202086:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.202218:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371149952:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302826.202274:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.202304:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc 0) 00000100:00100000:2.0:1789302826.202383:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:30117:x1876219350435200:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4109us (4326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.202393:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 342 00000100:00100000:0.0:1789302826.202698:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350435328 00000100:00100000:0.0:1789302826.202709:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350435328 00010000:00010000:0.0:1789302826.202752:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 11 pid: 13439 timeout: 763 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302826.202790:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 375 00000100:00100000:0.0:1789302826.202800:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350435328:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302826.202819:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302826.202847:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 11 pid: 13439 timeout: 763 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.202869:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 11 pid: 13439 timeout: 763 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.202908:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe43095184eac7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a03 expref: 11 pid: 13439 timeout: 763 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.202925:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302826.202931:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302826.202941:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.202953:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.202997:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302826.203015:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371150080:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302826.203105:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302826.207195:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371150080:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302826.207242:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302826.208659:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.208937:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350435456 00000100:00100000:2.0:1789302826.208960:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350435456 00002000:00100000:2.0:1789302826.209037:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1a:0x0] (0->157) req@ffff960f3f7ca300 x1876219350435456/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:402/0 lens 648/0 e 0 to 0 dl 1789302837 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302826.209052:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7ca300 with x1876219350435456 ext(0->157) 00002000:00010000:2.0:1789302826.209059:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f7ca300 00000100:00100000:2.0:1789302826.209095:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 111 00000100:00100000:2.0:1789302826.209107:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350435456:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302826.209863:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2345:x1876219350435456:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 752us (1207us total) trans 4294967367 rc 0/0 00000100:00100000:2.0:1789302826.209883:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 111 00002000:00100000:2.0:1789302826.209893:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1a:0x0] (0->157) req@ffff960f3f7ca300 x1876219350435456/t4294967367(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:402/0 lens 648/448 e 0 to 0 dl 1789302837 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302826.209902:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7ca300 with x1876219350435456 ext(0->157) 00002000:00010000:2.0:1789302826.209906:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f7ca300 00000100:00100000:1.0:1789302826.209994:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350435328:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 7192us (7920us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302826.210006:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 375 00000100:00100000:3.0:1789302826.212680:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.212833:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350435584 00000100:00100000:1.0:1789302826.212846:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350435584 00000100:00100000:1.0:1789302826.212925:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1253 00000100:00100000:1.0:1789302826.212937:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:30117:x1876219350435584:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:0.0:1789302826.249613:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968128 is committed 00000001:00080000:0.0:1789302826.249638:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968117 is committed 00000020:00080000:0.0:1789302826.249652:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 25 batchid updated 00000001:00080000:0.0:1789302826.249674:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968131 is committed 00000001:00080000:0.0:1789302826.249683:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968125 is committed 00000004:00080000:0.0:1789302826.249689:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 12 committed 00000020:00080000:2.0:1789302826.249764:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 25 00000020:00080000:2.0:1789302826.249780:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 25 00000100:00100000:1.0:1789302826.250011:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:30117:x1876219350435584:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 37067us (37336us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302826.250023:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1253 00000100:00100000:2.0:1789302826.251281:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000020:00080000:0.0:1789302826.251474:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f1d5b3480 refcount 2 committed 0 result 0 batchid 26 00000020:00080000:0.0:1789302826.251481:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10ea2500 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f1d5b3b40 00000020:00080000:0.0:1789302826.251485:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10ea2300 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f1daf6000 00000020:00080000:1.0:1789302826.251638:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 26 committed batchid 25 00000100:00100000:3.0:1789302826.252266:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302826.252306:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350435712 00000100:00100000:3.0:1789302826.252317:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350435712 00000100:00100000:3.0:1789302826.252379:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 343 00000100:00100000:3.0:1789302826.252392:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350435712:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000020:00080000:1.0:1789302826.253197:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 25 00000001:00080000:0.0:1789302826.253625:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968132 is committed 00000001:00080000:0.0:1789302826.253644:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968126 is committed 00000020:00080000:0.0:1789302826.253669:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f07e4e480 refcount 2 committed 0 result 0 batchid 27 00000020:00080000:0.0:1789302826.253675:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10e85300 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f07e4e600 00000020:00080000:0.0:1789302826.253683:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10e85900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f106c8000 00000020:00080000:0.0:1789302826.253716:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 27 committed batchid 25 00000020:00080000:0.0:1789302826.253764:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 25 00000100:00100000:2.0:1789302826.254138:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371150208:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789302826.254340:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302826.254566:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371150208 00000100:00100000:2.0:1789302826.254577:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371150208 00000100:00100000:2.0:1789302826.254630:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1789302826.254641:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3303:x1876219371150208:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1789302826.254687:0:13439:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:25 count 1 00002000:00010000:2.0:1789302826.254742:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x19:0x0] initial LVB size: 158, mtime: 0x6aa69826, atime: 0x6aa69827, ctime: 0x6aa69826, blocks: 0xb 00010000:00010000:2.0:1789302826.254753:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.254777:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=663s v=(0 0) 00010000:00010000:2.0:1789302826.254784:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x19:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302826.254792:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.254814:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.254826:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.254834:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.254855:0:13439:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789302826.255660:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+8:3303:x1876219371150208:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1019us (1317us total) trans 4294967370 rc 0/0 00000100:00100000:2.0:1789302826.255673:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00010000:00010000:2.0:1789302826.255705:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302826.255712:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.255722:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc16b18b0) 00000004:00080000:0.0:1789302826.255725:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f7cb480/2, rc 0, transno 4294967370 00010000:00010000:2.0:1789302826.255727:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302826.255733:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371150208:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1789302826.255737:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.255758:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.255767:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002a010bee/0x64fe43095184ead5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x19:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.255787:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302826.255788:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302826.255790:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302826.275569:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967367 is committed 00000100:00100000:1.0:1789302826.276039:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350435712:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 23638us (23768us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302826.276057:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 343 00000100:00100000:2.0:1789302826.281757:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.281889:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350435968 00000100:00100000:0.0:1789302826.281902:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350435968 00000100:00100000:0.0:1789302826.282060:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1254 00000100:00100000:0.0:1789302826.282073:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30117:x1876219350435968:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302826.282148:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302826.282186:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eac0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09fc expref: 47 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.282209:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302826.282215:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302826.282257:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.282276:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.282288:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302826.282596:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f6ccd2c0 reply state 0000000039c5474f transno 4294968133 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302826.282607:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302826.282738:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30117:x1876219350435968:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 667us (994us total) trans 4294968133 rc 0/0 00000100:00100000:0.0:1789302826.282751:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1254 00010000:00010000:1.0:1789302826.285787:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.285809:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.285818:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.285825:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302826.285831:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.285840:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.285861:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302826.285865:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302826.285882:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 2/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.285889:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eadc lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.286448:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.286663:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350436096 00000100:00100000:0.0:1789302826.286677:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350436096 00000100:00100000:0.0:1789302826.286745:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1255 00000100:00100000:0.0:1789302826.286758:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30117:x1876219350436096:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302826.286789:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302826.286815:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a11 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.286901:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.286918:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.286958:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:0.0:1789302826.286962:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302826.286966:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045000, rc 0) 00000100:00100000:0.0:1789302826.287042:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30117:x1876219350436096:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 286us (600us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302826.287050:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1255 00000100:00100000:2.0:1789302826.297732:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302826.297828:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350436224 00000100:00100000:3.0:1789302826.297843:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350436224 00000100:00100000:3.0:1789302826.297924:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:3.0:1789302826.297936:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350436224:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1789302826.298402:0:6696:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1789302826.298407:0:6696:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb mtime from disk: 0 -> 1789302822 00002000:00010000:3.0:1789302826.298409:0:6696:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb atime from disk: 0 -> 1789302824 00002000:00010000:3.0:1789302826.298412:0:6696:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb ctime from disk: 0 -> 1789302822 00002000:00010000:3.0:1789302826.298414:0:6696:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1a:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1789302826.298578:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350436224:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 642us (861us total) trans 4294967368 rc 0/0 00000100:00100000:3.0:1789302826.298588:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:3.0:1789302826.302320:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.303183:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350436480 00000100:00100000:1.0:1789302826.303209:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350436480 00000100:00100000:1.0:1789302826.303291:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1256 00000100:00100000:1.0:1789302826.303324:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30117:x1876219350436480:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1789302826.303422:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.303444:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.303456:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 4/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.303547:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.303567:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 48 pid: 18427 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302826.303585:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371150336:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302826.303719:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599a80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302826.307165:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371150336:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302826.307217:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302826.308490:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.309647:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350436608 00000100:00100000:2.0:1789302826.309667:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350436608 00010000:00010000:2.0:1789302826.309731:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 49 pid: 18427 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.309785:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 376 00000100:00100000:2.0:1789302826.309798:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09750c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350436608:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302826.309819:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302826.309827:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 49 pid: 18427 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.309843:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 49 pid: 18427 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.309882:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ea8f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x25:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09e7 expref: 49 pid: 18427 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.309901:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:2.0:1789302826.309908:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.309920:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.309930:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.309967:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302826.310047:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09750c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350436608:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 252us (1567us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.310056:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 376 00010000:00010000:1.0:1789302826.310294:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310303:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310354:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310382:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310389:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310395:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302826.310400:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310409:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310418:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302826.310421:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1789302826.310425:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 2/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.310430:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eaea lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302826.310481:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30117:x1876219350436480:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 7159us (8173us total) trans 0 rc -22/-22 00000100:00100000:1.0:1789302826.310490:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1256 00000100:00100000:2.0:1789302826.316885:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302826.318249:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350436736 00000100:00100000:3.0:1789302826.318265:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350436736 00000100:00100000:3.0:1789302826.318355:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 346 00000100:00100000:3.0:1789302826.318370:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350436736:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302826.318433:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.318461:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.318472:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302826.318478:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302826.318485:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.318751:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.318771:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 11 pid: 6697 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302826.318797:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7ca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371150464:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302826.318993:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302826.324159:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7ca680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371150464:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302826.324211:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.325608:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.325893:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350436864 00000100:00100000:1.0:1789302826.325907:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350436864 00010000:00010000:1.0:1789302826.325958:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 12 pid: 6697 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302826.326004:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 377 00000100:00100000:1.0:1789302826.326032:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350436864:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302826.326052:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302826.326083:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 12 pid: 6697 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.326099:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 12 pid: 6697 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.326127:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095184eace lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a0a expref: 12 pid: 6697 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.326147:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302826.326152:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302826.326161:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.326173:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.326213:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302826.326290:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350436864:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 260us (691us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302826.326298:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 377 00010000:00010000:2.0:1789302826.330070:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330100:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330166:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330176:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330187:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330195:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302826.330201:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330211:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330247:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.330256:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184eaf1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302826.330415:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350436736:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 12042us (13547us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.330427:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 346 00000100:00100000:2.0:1789302826.334463:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.334651:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350437120 00000100:00100000:0.0:1789302826.334666:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350437120 00000100:00100000:0.0:1789302826.334746:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1257 00000100:00100000:0.0:1789302826.334759:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30117:x1876219350437120:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302826.334867:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.334901:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.334916:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.335623:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.335640:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.335650:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 4/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.335702:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.335718:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 48 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302826.335735:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371150592:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302826.335900:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4d80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302826.340301:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302826.340337:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371150592:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302826.340447:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302826.340635:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350437248 00000100:00100000:1.0:1789302826.340645:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350437248 00010000:00010000:1.0:1789302826.340697:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 49 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302826.340767:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 378 00000100:00100000:1.0:1789302826.340779:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350437248:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302826.340806:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302826.340815:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 49 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.340828:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 49 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.340856:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eae3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a11 expref: 49 pid: 6919 timeout: 763 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.340884:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:1.0:1789302826.340890:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,1 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.340900:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.340909:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.340943:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302826.341049:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2347:x1876219350437248:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 272us (765us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302826.341059:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 378 00010000:00010000:0.0:1789302826.341109:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.341116:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302826.341772:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968134 00000001:00080000:0.0:1789302826.341959:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968135 00000001:00080000:0.0:1789302826.342090:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968136 00000100:00100000:0.0:1789302826.342197:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371150720:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302826.342274:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302826.342363:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302826.342443:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371150720 00000100:00100000:2.0:1789302826.342454:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371150720 00000100:00100000:2.0:1789302826.342695:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 59 00000100:00100000:2.0:1789302826.342707:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371150720:12345-0@lo:1000:mdt00_001.0 00000100:00080000:1.0:1789302826.342737:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302826.342828:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371150720:12345-0@lo:1000:mdt00_001.0 Request processed in 123us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.342836:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 59 00000100:00100000:3.0:1789302826.342926:0:5844:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967366 gen 2 00000004:00080000:3.0:1789302826.342934:0:5844:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967365 committed transno 4294967366 00000004:00080000:3.0:1789302826.342956:0:5844:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f025500, transno 4294967366 00000100:00100000:3.0:1789302826.342978:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025f80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371150720:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1789302826.344026:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968137 00000100:00100000:3.0:1789302826.344126:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371150848:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789302826.344221:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302826.344249:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371150848 00000100:00100000:3.0:1789302826.344252:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302826.344258:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371150848 00000100:00100000:0.0:1789302826.344304:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 60 00000100:00100000:0.0:1789302826.344313:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371150848:12345-0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302826.344921:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5844:x1876219371150848:12345-0@lo:1000:mdt00_001.0 Request processed in 605us (704us total) trans 4294967369 rc 0/0 00000100:00100000:0.0:1789302826.344930:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 60 00000100:00100000:3.0:1789302826.345096:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cbb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371150848:0@lo:1000:mdt00_001.0 00000001:00080000:3.0:1789302826.345812:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968138 00010000:00010000:3.0:1789302826.345867:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.345890:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.345899:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.345906:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302826.345912:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.345927:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346017:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f03927100) 00010000:00010000:3.0:1789302826.346021:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302826.346027:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 2/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346034:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184eaff lrc: 0/0,0 mode: --/EX res: [0x200000402:0x25:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346076:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346082:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346089:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346094:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302826.346098:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346104:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346113:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927400) 00010000:00010000:3.0:1789302826.346116:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302826.346126:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d2 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346156:0:5844:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d2 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346360:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d2 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302826.346478:0:5844:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3800/0x64fe43095184ea34 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a09d2 expref: 96 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302826.346497:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371150976:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302826.346682:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08c43e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302826.349192:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371150976:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302826.349432:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349447:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184ea2d lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349497:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349555:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349569:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349583:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349590:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349597:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349605:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302826.349609:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349623:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349637:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927400) 00010000:00010000:0.0:1789302826.349641:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302826.349645:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349651:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb06 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x23:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349663:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349669:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349676:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349681:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302826.349684:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349691:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349698:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x23:0x0].0x0 (ffff960f03927400) 00010000:00010000:0.0:1789302826.349701:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302826.349703:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302826.349714:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eaf8 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x23:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302826.350040:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:30117:x1876219350437120:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 15280us (15589us total) trans 4294968139 rc 0/0 00000100:00100000:0.0:1789302826.350054:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1257 00000020:00080000:0.0:1789302826.350063:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:1.0:1789302826.350161:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302826.350167:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302826.350915:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:1.0:1789302826.350929:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302826.353683:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302826.353775:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350437504 00000100:00100000:3.0:1789302826.353790:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350437504 00000100:00100000:3.0:1789302826.353854:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 254 00000100:00100000:3.0:1789302826.353865:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:30117:x1876219350437504:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1789302826.356820:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x25:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1789302826.356970:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968140 00010000:00010000:2.0:1789302826.357584:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.357607:0:12976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.357623:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.357634:0:12976:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.357640:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.357646:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.357673:0:12976:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789302826.357844:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30117:x1876219350437504:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 3974us (4172us total) trans 4294968141 rc 0/0 00000100:00100000:2.0:1789302826.357856:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 254 00000020:00080000:2.0:1789302826.357864:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1789302826.358293:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302826.358303:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.358312:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302826.358316:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.358326:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.358343:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x25:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:1.0:1789302826.358347:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789302826.358353:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 1/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.358360:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184eb0d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x25:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302826.358475:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302826.358478:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302826.358480:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302826.359125:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302826.359251:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350437632 00000100:00100000:1.0:1789302826.359275:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350437632 00000100:00100000:1.0:1789302826.359347:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 255 00000100:00100000:1.0:1789302826.359359:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076eb050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30113:x1876219350437632:12345-192.168.203.11@tcp:49:cat.0 00000100:00100000:1.0:1789302826.359558:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076eb050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30113:x1876219350437632:12345-192.168.203.11@tcp:49:cat.0 Request processed in 200us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302826.359586:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 255 00000100:00100000:3.0:1789302826.367324:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.367652:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350437760 00000100:00100000:2.0:1789302826.367667:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350437760 00000100:00100000:2.0:1789302826.367769:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:2.0:1789302826.367788:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+11:30113:x1876219350437760:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302826.367816:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302826.367848:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000754ce84f/0x64fe43095184eb14 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a18 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302826.367892:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1a:0x0] initial LVB size: 158, mtime: 0x6aa69826, atime: 0x6aa69828, ctime: 0x6aa69826, blocks: 0xb 00010000:00010000:2.0:1789302826.367923:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=663s v=(0 0) 00010000:00010000:2.0:1789302826.367929:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302826.367949:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000754ce84f/0x64fe43095184eb14 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a18 expref: 12 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.367977:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000754ce84f/0x64fe43095184eb14 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a18 expref: 12 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302826.367994:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3200, rc 0) 00000100:00080000:1.0:1789302826.368195:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302826.368237:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:30113:x1876219350437760:12345-192.168.203.11@tcp:101:cat.0 Request processed in 451us (927us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302826.368245:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 347 00000100:00100000:0.0:1789302826.373453:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302826.373655:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350437888 00000100:00100000:3.0:1789302826.373669:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350437888 00002000:00100000:3.0:1789302826.373734:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1a:0x0] (0->4095) req@ffff960f3f027100 x1876219350437888/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:402/0 lens 488/0 e 0 to 0 dl 1789302837 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302826.373747:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f027100 with x1876219350437888 ext(0->4095) 00002000:00010000:3.0:1789302826.373754:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f027100 00000100:00100000:3.0:1789302826.373800:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 112 00000100:00100000:3.0:1789302826.373810:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350437888:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:3.0:1789302826.374269:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350437888:12345-192.168.203.11@tcp:3:cat.0 Request processed in 458us (829us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302826.374278:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 112 00002000:00100000:3.0:1789302826.374287:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1a:0x0] (0->4095) req@ffff960f3f027100 x1876219350437888/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:402/0 lens 488/4536 e 0 to 0 dl 1789302837 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302826.374296:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f027100 with x1876219350437888 ext(0->4095) 00002000:00010000:3.0:1789302826.374300:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f027100 00000100:00100000:0.0:1789302826.380305:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302826.380715:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350438144 00000100:00100000:2.0:1789302826.380732:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350438144 00000100:00100000:2.0:1789302826.380819:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 256 00000100:00100000:2.0:1789302826.380832:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30113:x1876219350438144:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:2.0:1789302826.381194:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30113:x1876219350438144:12345-192.168.203.11@tcp:35:cat.0 Request processed in 362us (900us total) trans 4294968142 rc 0/0 00000100:00100000:2.0:1789302826.381204:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 256 00010000:00010000:2.0:1789302826.638461:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302827.023452:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371151488:0@lo:400:kworker.0 00000100:00100000:3.0:1789302827.023569:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302827.023651:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1789302827.023657:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371151104:0@lo:400:kworker.0 00000100:00100000:3.0:1789302827.023701:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302827.023727:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371151616:0@lo:400:kworker.0 00000100:00100000:3.0:1789302827.023754:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302827.023788:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371151104 00000100:00100000:3.0:1789302827.023797:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371151104 00000100:00100000:3.0:1789302827.023862:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 584 00000100:00100000:3.0:1789302827.023882:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371151104:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302827.023942:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371151104:12345-0@lo:400:kworker.0 Request processed in 62us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302827.023949:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 584 00000100:00100000:3.0:1789302827.024179:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371151104:0@lo:400:kworker.0 00000100:00100000:1.0:1789302827.024317:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371151488 00000100:00100000:1.0:1789302827.024323:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371151488 00000100:00100000:1.0:1789302827.024376:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:1.0:1789302827.024384:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371151488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302827.024432:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371151488:12345-0@lo:400:kworker.0 Request processed in 49us (863us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302827.024439:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:1.0:1789302827.024469:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371151488:0@lo:400:kworker.0 00000100:00100000:2.0:1789302827.024760:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371151616 00000100:00100000:2.0:1789302827.024766:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371151616 00000100:00100000:2.0:1789302827.024829:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1789302827.024838:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371151616:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302827.024892:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371151616:12345-0@lo:400:kworker.0 Request processed in 56us (1136us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.024898:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:0.0:1789302827.025452:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371151232:0@lo:400:kworker.0 00000100:00100000:0.0:1789302827.025497:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302827.025566:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371151744:0@lo:400:kworker.0 00000100:00100000:0.0:1789302827.025593:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302827.025622:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302827.025629:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371151360:0@lo:400:kworker.0 00000100:00100000:0.0:1789302827.025651:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302827.025701:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371151744 00000100:00100000:0.0:1789302827.025708:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371151744 00000100:00100000:0.0:1789302827.025756:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:0.0:1789302827.025766:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+8:3300:x1876219371151744:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302827.025810:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+8:3300:x1876219371151744:12345-0@lo:400:kworker.0 Request processed in 47us (216us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302827.025817:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:0.0:1789302827.025864:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa7800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371151744:0@lo:400:kworker.0 00000100:00080000:1.0:1789302827.026276:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302827.026283:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302827.026315:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371151616:0@lo:400:kworker.0 00000100:00100000:2.0:1789302827.026706:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371151232 00000100:00100000:2.0:1789302827.026714:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371151232 00000100:00100000:2.0:1789302827.026762:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371151360 00000100:00100000:2.0:1789302827.026766:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371151360 00000100:00100000:2.0:1789302827.026804:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789302827.026812:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371151232:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302827.026897:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371151232:12345-0@lo:400:kworker.0 Request processed in 86us (1399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.026903:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1789302827.026922:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:2.0:1789302827.026927:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371151360:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302827.026951:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371151360:12345-0@lo:400:kworker.0 Request processed in 25us (1299us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.026956:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00080000:1.0:1789302827.028129:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302827.028557:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371151360:0@lo:400:kworker.0 00000100:00100000:0.0:1789302827.028580:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371151232:0@lo:400:kworker.0 00000100:00100000:2.0:1789302827.278678:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302827.278744:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302827.278782:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302827.278826:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302827.278847:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302827.278884:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302827.278914:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302827.278952:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350438656 00000100:00100000:2.0:1789302827.278968:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350438656 00000100:00100000:2.0:1789302827.279043:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350438912 00000100:00100000:2.0:1789302827.279046:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350438912 00000100:00100000:2.0:1789302827.279077:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350438528 00000100:00100000:2.0:1789302827.279080:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350438528 00000100:00100000:2.0:1789302827.279090:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439040 00000100:00100000:2.0:1789302827.279092:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439040 00000100:00100000:2.0:1789302827.279109:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:2.0:1789302827.279120:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350438656:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302827.279226:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350438656:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 107us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.279235:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:2.0:1789302827.279255:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:2.0:1789302827.279263:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350438912:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302827.279290:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350438912:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (507us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.279297:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:2.0:1789302827.279308:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:2.0:1789302827.279313:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350438528:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302827.279335:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2345:x1876219350438528:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.279342:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:2.0:1789302827.279354:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:2.0:1789302827.279360:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350439040:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302827.279384:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350439040:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (469us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.279389:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:2.0:1789302827.279409:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350438400 00000100:00100000:2.0:1789302827.279413:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350438400 00000100:00100000:2.0:1789302827.279440:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350438784 00000100:00100000:2.0:1789302827.279444:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350438784 00000100:00100000:2.0:1789302827.279463:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:2.0:1789302827.279469:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:2345:x1876219350438400:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302827.279543:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:2345:x1876219350438400:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 74us (882us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.279549:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:2.0:1789302827.279562:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00100000:2.0:1789302827.279568:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2345:x1876219350438784:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302827.279593:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2345:x1876219350438784:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (709us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302827.279599:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00080000:2.0:1789302827.279621:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302827.280014:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350438272 00000100:00100000:3.0:1789302827.280021:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350438272 00000100:00100000:3.0:1789302827.280070:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 585 00000100:00100000:3.0:1789302827.280078:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350438272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302827.280138:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350438272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 50us (1300us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302827.280146:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 585 00000100:00100000:1.0:1789302827.544787:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371151872:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302827.544891:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302827.545610:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371151872 00000100:00100000:3.0:1789302827.545622:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371151872 00000100:00100000:3.0:1789302827.545692:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1789302827.545704:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3301:x1876219371151872:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302827.545754:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302827.545823:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3301:x1876219371151872:12345-0@lo:13:osp-pre-1-0.0 Request processed in 121us (931us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302827.545831:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:1.0:1789302827.547452:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371151872:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302827.661067:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302828.045067:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302828.045100:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371152000:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302828.045192:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302828.045617:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152000 00000100:00100000:2.0:1789302828.045628:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152000 00000100:00100000:2.0:1789302828.045719:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 195 00000100:00100000:2.0:1789302828.045733:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371152000:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302828.045780:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85000192 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302828.045842:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371152000:12345-0@lo:13:osp-pre-0-0.0 Request processed in 112us (649us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302828.045849:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 195 00000100:00080000:0.0:1789302828.046650:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302828.046719:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371152000:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1789302828.692423:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302829.709839:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302830.178062:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967369 is committed 00000001:00080000:2.0:1789302830.178092:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967367 is committed 00000001:00080000:2.0:1789302830.178101:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967370 is committed 00000040:00080000:0.0:1789302830.540810:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302830.540822:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302830.541223:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 37 count 4 00000040:00080000:0.0:1789302830.541235:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302830.735661:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302830.972894:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302830.972982:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439168 00000100:00100000:1.0:1789302830.972995:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439168 00000100:00100000:1.0:1789302830.974706:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1258 00000100:00100000:1.0:1789302830.974725:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30075:x1876219350439168:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1789302830.974874:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:30075:x1876219350439168:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 155us (1994us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302830.974883:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1258 00000100:00080000:0.0:1789302830.978906:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000001:00080000:0.0:1789302831.728618:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968137 is committed 00000020:00080000:0.0:1789302831.728649:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 27 batchid updated 00000020:00080000:0.0:1789302831.728656:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 26 batchid updated 00000001:00080000:0.0:1789302831.728680:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968142 is committed 00000001:00080000:0.0:1789302831.728687:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968141 is committed 00000020:00080000:0.0:1789302831.728711:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f3f73a780 refcount 2 committed 0 result 0 batchid 28 00000020:00080000:0.0:1789302831.728718:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10e85c00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f3f73ad80 00000020:00080000:0.0:1789302831.728721:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f10e85700 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3f7b4400 00000020:00080000:3.0:1789302831.729039:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 28 committed batchid 27 00000020:00080000:3.0:1789302831.729315:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 27 00000001:00080000:2.0:1789302831.741596:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967369 is committed 00000001:00080000:2.0:1789302831.741628:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967368 is committed 00010000:00010000:2.0:1789302831.759226:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302832.141884:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302832.141908:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371152128:0@lo:400:kworker.0 00000100:00100000:3.0:1789302832.141977:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302832.142035:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371152512:0@lo:400:kworker.0 00000100:00100000:3.0:1789302832.142060:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302832.142076:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302832.142081:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371152640:0@lo:400:kworker.0 00000100:00100000:3.0:1789302832.142100:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302832.142169:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152128 00000100:00100000:2.0:1789302832.142178:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152128 00000100:00100000:2.0:1789302832.142229:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:2.0:1789302832.142238:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371152128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302832.142292:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371152128:12345-0@lo:400:kworker.0 Request processed in 54us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302832.142298:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:2.0:1789302832.142330:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152512 00000100:00100000:2.0:1789302832.142334:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152512 00000100:00100000:2.0:1789302832.142366:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:2.0:1789302832.142372:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371152512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302832.142398:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371152512:12345-0@lo:400:kworker.0 Request processed in 27us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302832.142403:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:2.0:1789302832.142417:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152640 00000100:00100000:2.0:1789302832.142420:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152640 00000100:00100000:2.0:1789302832.142442:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:2.0:1789302832.142447:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371152640:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302832.142470:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371152640:12345-0@lo:400:kworker.0 Request processed in 23us (369us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302832.142475:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00080000:2.0:1789302832.142487:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302832.142491:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302832.142544:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371152640:0@lo:400:kworker.0 00000100:00100000:2.0:1789302832.142551:0:3302:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967369 gen 2 00000004:00080000:2.0:1789302832.142556:0:3302:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967369 committed transno 4294967369 00000100:00100000:2.0:1789302832.142575:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371152512:0@lo:400:kworker.0 00000100:00100000:2.0:1789302832.142583:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371152128:0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.142674:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371152256:0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.142704:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302832.142725:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371152768:0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.142745:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302832.142764:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302832.142769:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371152384:0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.142787:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302832.142815:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152256 00000100:00100000:1.0:1789302832.142819:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152256 00000100:00100000:1.0:1789302832.142857:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152384 00000100:00100000:1.0:1789302832.142859:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152384 00000100:00100000:1.0:1789302832.142880:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:1.0:1789302832.142885:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371152256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.142919:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371152256:12345-0@lo:400:kworker.0 Request processed in 33us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302832.142925:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:1.0:1789302832.142937:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:1.0:1789302832.142944:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371152384:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.142968:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371152384:12345-0@lo:400:kworker.0 Request processed in 25us (180us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302832.142973:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:1.0:1789302832.142989:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152768 00000100:00100000:1.0:1789302832.142992:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152768 00000100:00100000:1.0:1789302832.143014:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:1.0:1789302832.143020:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371152768:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.143046:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371152768:12345-0@lo:400:kworker.0 Request processed in 26us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302832.143053:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00080000:1.0:1789302832.143073:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302832.143076:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302832.143092:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371152384:0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.143101:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371152256:0@lo:400:kworker.0 00000100:00100000:1.0:1789302832.143107:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967370 gen 2 00000004:00080000:1.0:1789302832.143111:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967368 committed transno 4294967370 00000004:00080000:1.0:1789302832.143124:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967369 committed transno 4294967370 00000004:00080000:1.0:1789302832.143135:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f7cb480, transno 4294967370 00000100:00100000:1.0:1789302832.143145:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371152768:0@lo:400:kworker.0 00000100:00100000:0.0:1789302832.402208:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302832.402251:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302832.402287:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302832.402309:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302832.402328:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302832.402347:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302832.402370:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302832.402398:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439808 00000100:00100000:0.0:1789302832.402408:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439808 00000100:00100000:0.0:1789302832.402472:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439424 00000100:00100000:0.0:1789302832.402475:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439424 00000100:00100000:0.0:1789302832.402499:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:0.0:1789302832.402530:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2348:x1876219350439808:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302832.402552:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439296 00000100:00100000:2.0:1789302832.402556:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439296 00000100:00100000:2.0:1789302832.402579:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 587 00000100:00100000:2.0:1789302832.402586:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350439296:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302832.402602:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2348:x1876219350439808:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 75us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302832.402609:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350439296:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302832.402610:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:2.0:1789302832.402614:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 587 00000100:00100000:0.0:1789302832.402625:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:0.0:1789302832.402631:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:2348:x1876219350439424:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302832.402653:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:2348:x1876219350439424:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302832.402659:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:0.0:1789302832.402684:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439552 00000100:00100000:0.0:1789302832.402688:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439552 00000100:00100000:0.0:1789302832.402724:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440064 00000100:00100000:0.0:1789302832.402727:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440064 00000100:00100000:0.0:1789302832.402746:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439680 00000100:00100000:0.0:1789302832.402749:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439680 00000100:00100000:0.0:1789302832.402760:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350439936 00000100:00100000:0.0:1789302832.402762:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350439936 00000100:00100000:0.0:1789302832.402776:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 462 00000100:00100000:0.0:1789302832.402782:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350439552:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302832.402807:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350439552:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (555us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302832.402812:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 462 00000100:00100000:0.0:1789302832.402839:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:0.0:1789302832.402847:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350440064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302832.402871:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350440064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (561us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302832.402876:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:0.0:1789302832.402888:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00100000:0.0:1789302832.402894:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350439680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302832.402915:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350439680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302832.402921:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00100000:0.0:1789302832.402931:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 465 00000100:00100000:0.0:1789302832.402938:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350439936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302832.402960:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350439936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (589us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302832.402965:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 465 00000100:00080000:0.0:1789302832.402991:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302832.402995:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302832.402997:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302832.652822:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302832.652853:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371152896:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302832.652918:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302832.652982:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371152896 00000100:00100000:3.0:1789302832.652990:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371152896 00000100:00100000:3.0:1789302832.653038:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:3.0:1789302832.653047:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371152896:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302832.653079:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 83296256 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302832.653122:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371152896:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (202us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302832.653129:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:3.0:1789302832.653173:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371152896:0@lo:13:osp-pre-1-0.0 00000001:02000400:0.0:1789302832.775576:0:21461:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12q: file attributes are refreshed after restore ========================================================== 08:33:50 (1789302830) 00010000:00010000:2.0:1789302832.780727:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302833.003486:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.003646:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440192 00000100:00100000:3.0:1789302833.003657:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440192 00000100:00100000:3.0:1789302833.003713:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00100000:3.0:1789302833.003723:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:30710:x1876219350440192:12345-192.168.203.11@tcp:101:mount.lustre.0 00010000:00010000:3.0:1789302833.003741:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302833.003761:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f3fcf0200/0x64fe43095184eb1b lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a1f expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.003793:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f3fcf0200/0x64fe43095184eb1b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a1f expref: 14 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.003805:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f3fcf0200/0x64fe43095184eb1b lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a1f expref: 14 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.003814:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302833.003888:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30710:x1876219350440192:12345-192.168.203.11@tcp:101:mount.lustre.0 Request processed in 166us (412us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.005319:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00100000:2.0:1789302833.015653:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.015717:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440320 00000100:00100000:3.0:1789302833.015727:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440320 00000100:00100000:3.0:1789302833.015781:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 589 00000100:00100000:3.0:1789302833.015791:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30710:x1876219350440320:12345-192.168.203.11@tcp:501:mount.lustre.0 00000100:00100000:3.0:1789302833.016226:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30710:x1876219350440320:12345-192.168.203.11@tcp:501:mount.lustre.0 Request processed in 435us (580us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.016235:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 589 00000100:00100000:1.0:1789302833.018109:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.018267:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440448 00000100:00100000:3.0:1789302833.018275:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440448 00000100:00100000:3.0:1789302833.018359:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 590 00000100:00100000:3.0:1789302833.018369:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30710:x1876219350440448:12345-192.168.203.11@tcp:503:mount.lustre.0 00000100:00100000:3.0:1789302833.018932:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30710:x1876219350440448:12345-192.168.203.11@tcp:503:mount.lustre.0 Request processed in 563us (830us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.018940:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 590 00000100:00100000:0.0:1789302833.021186:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.021253:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440576 00000100:00100000:2.0:1789302833.021267:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440576 00000100:00100000:2.0:1789302833.021331:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 591 00000100:00100000:2.0:1789302833.021352:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30714:x1876219350440576:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302833.021749:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30714:x1876219350440576:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 398us (575us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.021758:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 591 00000100:00100000:0.0:1789302833.035780:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.035874:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440704 00000100:00100000:3.0:1789302833.035889:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440704 00000100:00100000:3.0:1789302833.035955:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 592 00000100:00100000:3.0:1789302833.035964:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30714:x1876219350440704:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:3.0:1789302833.036395:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30714:x1876219350440704:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 430us (630us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.036403:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 592 00000100:00100000:0.0:1789302833.042772:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.042849:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440832 00000100:00100000:3.0:1789302833.042863:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440832 00000100:00100000:3.0:1789302833.043030:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 593 00000100:00100000:3.0:1789302833.043040:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30714:x1876219350440832:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:3.0:1789302833.043318:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30714:x1876219350440832:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 278us (560us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.043326:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 593 00000100:00100000:3.0:1789302833.051079:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.051174:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350440960 00000100:00100000:1.0:1789302833.051194:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350440960 00000100:00100000:1.0:1789302833.051291:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 594 00000100:00100000:1.0:1789302833.051305:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:30710:x1876219350440960:12345-192.168.203.11@tcp:101:mount.lustre.0 00010000:00010000:1.0:1789302833.051328:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302833.051354:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960f07ef6600/0x64fe43095184eb22 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a49 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.051396:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960f07ef6600/0x64fe43095184eb22 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a49 expref: 15 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.051411:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960f07ef6600/0x64fe43095184eb22 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a49 expref: 15 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.051422:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc 0) 00000100:00100000:1.0:1789302833.051556:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+15:30710:x1876219350440960:12345-192.168.203.11@tcp:101:mount.lustre.0 Request processed in 254us (485us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302833.051564:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 594 00000100:00100000:1.0:1789302833.056971:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.057491:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441088 00000100:00100000:3.0:1789302833.057614:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441088 00000100:00100000:3.0:1789302833.057723:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 595 00000100:00100000:3.0:1789302833.057735:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+15:30710:x1876219350441088:12345-192.168.203.11@tcp:256:mount.lustre.0 20000000:01000000:3.0:1789302833.057760:0:5838:0:(mgs_nids.c:915:mgs_get_ir_logs()) Reading IR log lustre-cliir bufsize 1048576. 20000000:01000000:3.0:1789302833.057784:0:5838:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4052/1/256/255. 20000000:01000000:3.0:1789302833.057789:0:5838:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 4008/1/256/255. 20000000:01000000:3.0:1789302833.057792:0:5838:0:(mgs_nids.c:307:mgs_nidtbl_read()) fsname lustre, entry size 44, pages 3964/1/256/255. 20000000:01000000:3.0:1789302833.057794:0:5838:0:(mgs_nids.c:318:mgs_nidtbl_read()) Read IR logs lustre return with 132, version 5 00000100:00100000:3.0:1789302833.060451:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+15:30710:x1876219350441088:12345-192.168.203.11@tcp:256:mount.lustre.0 Request processed in 2707us (3478us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.060464:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 595 00000100:00100000:1.0:1789302833.069624:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.073078:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441216 00000100:00100000:3.0:1789302833.073092:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441216 00000100:00100000:2.0:1789302833.077105:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 596 00000100:00100000:2.0:1789302833.077120:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+15:30710:x1876219350441216:12345-192.168.203.11@tcp:101:mount.lustre.0 00010000:00010000:2.0:1789302833.077136:0:5836:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302833.077401:0:5836:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: MGS lock: ffff960e028b2200/0x64fe43095184eb29 lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a50 expref: -99 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.077440:0:5836:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: MGS lock: ffff960e028b2200/0x64fe43095184eb29 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a50 expref: 16 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.077451:0:5836:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: MGS lock: ffff960e028b2200/0x64fe43095184eb29 lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a50 expref: 16 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.077460:0:5836:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2200, rc 0) 00000100:00100000:2.0:1789302833.077547:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30710:x1876219350441216:12345-192.168.203.11@tcp:101:mount.lustre.0 Request processed in 429us (7931us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.077555:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 596 00000100:00100000:1.0:1789302833.080589:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.080676:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441344 00000100:00100000:3.0:1789302833.080686:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441344 00000100:00100000:3.0:1789302833.080737:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 597 00000100:00100000:3.0:1789302833.080747:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30710:x1876219350441344:12345-192.168.203.11@tcp:501:mount.lustre.0 00000100:00100000:3.0:1789302833.085545:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30710:x1876219350441344:12345-192.168.203.11@tcp:501:mount.lustre.0 Request processed in 4788us (4953us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.085561:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 597 00000100:00100000:0.0:1789302833.097626:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.097808:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441472 00000100:00100000:2.0:1789302833.097818:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441472 00000100:00100000:2.0:1789302833.097949:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 598 00000100:00100000:2.0:1789302833.097961:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30710:x1876219350441472:12345-192.168.203.11@tcp:503:mount.lustre.0 00000100:00100000:2.0:1789302833.099932:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30710:x1876219350441472:12345-192.168.203.11@tcp:503:mount.lustre.0 Request processed in 1965us (2307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.099945:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 598 00000100:00100000:1.0:1789302833.125758:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.140409:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441600 00000100:00100000:3.0:1789302833.140424:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441600 00000100:00100000:2.0:1789302833.144594:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 599 00000100:00100000:2.0:1789302833.144610:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30717:x1876219350441600:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302833.144939:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30717:x1876219350441600:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 332us (19189us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.144947:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 599 00000100:00100000:1.0:1789302833.152341:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.152572:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441728 00000100:00100000:2.0:1789302833.152582:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441728 00000100:00100000:2.0:1789302833.153091:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 600 00000100:00100000:2.0:1789302833.153102:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30717:x1876219350441728:12345-192.168.203.11@tcp:502:llog_process_th.0 00000100:00100000:2.0:1789302833.153364:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:30717:x1876219350441728:12345-192.168.203.11@tcp:502:llog_process_th.0 Request processed in 264us (1040us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.153372:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 600 00000100:00100000:1.0:1789302833.160432:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.160767:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441856 00000100:00100000:2.0:1789302833.160777:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441856 00000100:00100000:2.0:1789302833.160836:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1259 00000100:00100000:2.0:1789302833.160845:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2344:x1876219350441856:12345-192.168.203.11@tcp:38:mount.lustre.0 00010000:00080000:2.0:1789302833.160989:0:18427:0:(ldlm_lib.c:1453:target_handle_connect()) lustre-MDT0000: connection from 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0@192.168.203.11@tcp t0 exp 0000000000000000 cur 670 last 0 00000020:00000080:2.0:1789302833.161039:0:18427:0:(genops.c:1384:class_connect()) connect: client 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0, cookie 0x64fe43095184eb30 00000004:00100000:2.0:1789302833.161071:0:18427:0:(mdt_handler.c:7293:mdt_connect_internal()) lustre-MDT0000: cli supports cksum type f7, return f7 00000100:00100000:1.0:1789302833.162172:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.162265:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350442368 00000100:00100000:3.0:1789302833.162273:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350442368 00000100:00100000:3.0:1789302833.162324:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 466 00000100:00100000:3.0:1789302833.162333:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350442368:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302833.162399:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2347:x1876219350442368:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 67us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.162407:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 466 00000100:00080000:2.0:1789302833.164637:0:18427:0:(import.c:65:import_set_state_nolock()) ffff960f10083000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789302833.164712:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2344:x1876219350441856:12345-192.168.203.11@tcp:38:mount.lustre.0 Request processed in 3866us (4287us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.164724:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1259 00000100:00100000:3.0:1789302833.165026:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.165135:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.165165:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.165190:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.165213:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350442624 00000100:00100000:3.0:1789302833.165220:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350442624 00000100:00100000:3.0:1789302833.165303:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350442240 00000100:00100000:3.0:1789302833.165308:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350442240 00000100:00100000:3.0:1789302833.165328:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:3.0:1789302833.165337:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2347:x1876219350442624:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302833.165377:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2347:x1876219350442624:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 41us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.165384:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:3.0:1789302833.165399:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:3.0:1789302833.165405:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:2347:x1876219350442240:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302833.165424:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:2347:x1876219350442240:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 19us (258us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.165429:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:3.0:1789302833.165445:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350442880 00000100:00100000:3.0:1789302833.165448:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350442880 00000100:00100000:3.0:1789302833.165467:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350442496 00000100:00100000:3.0:1789302833.165469:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350442496 00000100:00100000:3.0:1789302833.165485:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 467 00000100:00100000:3.0:1789302833.165490:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350442880:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302833.165546:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350442880:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 55us (411us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.165552:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 467 00000100:00100000:3.0:1789302833.165683:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 468 00000100:00100000:3.0:1789302833.165692:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350442496:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302833.165718:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350442496:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (526us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.165723:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 468 00000100:00100000:3.0:1789302833.166845:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.167087:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350442752 00000100:00100000:3.0:1789302833.167094:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350442752 00000100:00100000:3.0:1789302833.167141:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 469 00000100:00100000:3.0:1789302833.167149:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350442752:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302833.167193:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350442752:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 45us (353us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.167245:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 469 00000100:00100000:2.0:1789302833.167699:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371153024:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302833.167745:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302833.167986:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371153024 00000100:00100000:3.0:1789302833.167992:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371153024 00000100:00100000:3.0:1789302833.168027:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 197 00000100:00100000:3.0:1789302833.168035:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371153024:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302833.168065:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85000192 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302833.168111:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371153024:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (365us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.168117:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 197 00000100:00100000:0.0:1789302833.168194:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa5500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371153024:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302833.171932:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.172027:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350441984 00000100:00100000:2.0:1789302833.172038:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350441984 00000100:00100000:2.0:1789302833.172116:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1260 00000100:00100000:2.0:1789302833.172129:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:0:x1876219350441984:12345-192.168.203.11@tcp:41:mount.lustre.0 00000004:00000024:2.0:1789302833.172178:0:18427:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 18087936 pending 0 free 2259681280 avail 2257584128 00000100:00100000:2.0:1789302833.172245:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:0:x1876219350441984:12345-192.168.203.11@tcp:41:mount.lustre.0 Request processed in 119us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.172256:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1260 00000100:00080000:0.0:1789302833.172330:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302833.172867:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.173141:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350442112 00000100:00100000:3.0:1789302833.173149:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350442112 00000100:00100000:3.0:1789302833.173189:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1261 00000100:00100000:3.0:1789302833.173198:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30710:x1876219350442112:12345-192.168.203.11@tcp:41:mount.lustre.0 00000004:00000024:3.0:1789302833.173218:0:5843:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 18087936 pending 0 free 2259681280 avail 2257584128 00000100:00100000:3.0:1789302833.173271:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30710:x1876219350442112:12345-192.168.203.11@tcp:41:mount.lustre.0 Request processed in 73us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.173278:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1261 00000100:00100000:1.0:1789302833.186996:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302833.187085:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350443008 00000100:00100000:0.0:1789302833.187104:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350443008 00000100:00100000:0.0:1789302833.187197:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1262 00000100:00100000:0.0:1789302833.187209:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30710:x1876219350443008:12345-192.168.203.11@tcp:40:mount.lustre.0 00000100:00100000:0.0:1789302833.187368:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30710:x1876219350443008:12345-192.168.203.11@tcp:40:mount.lustre.0 Request processed in 160us (390us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302833.187376:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1262 00000100:00100000:0.0:1789302833.194947:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302833.194992:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302833.195020:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.195131:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350443392 00000100:00100000:1.0:1789302833.195143:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350443392 00000100:00100000:1.0:1789302833.195200:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 257 00000100:00100000:1.0:1789302833.195210:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30710:x1876219350443392:12345-192.168.203.11@tcp:33:mount.lustre.0 00000004:00200000:1.0:1789302833.195288:0:5847:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000100:00100000:1.0:1789302833.195363:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30710:x1876219350443392:12345-192.168.203.11@tcp:33:mount.lustre.0 Request processed in 155us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302833.195371:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 257 00000100:00100000:2.0:1789302833.195655:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350443136 00000100:00100000:2.0:1789302833.195661:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350443136 00000100:00100000:2.0:1789302833.195692:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350443264 00000100:00100000:2.0:1789302833.195696:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350443264 00000100:00100000:2.0:1789302833.195716:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 348 00000100:00100000:2.0:1789302833.195721:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2344:x1876219350443136:12345-192.168.203.11@tcp:8:mount.lustre.0 00010000:00080000:2.0:1789302833.195759:0:6696:0:(ldlm_lib.c:1453:target_handle_connect()) lustre-OST0000: connection from 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0@192.168.203.11@tcp t0 exp 0000000000000000 cur 670 last 0 00000020:00000080:2.0:1789302833.195801:0:6696:0:(genops.c:1384:class_connect()) connect: client 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0, cookie 0x64fe43095184eb37 00002000:00100000:2.0:1789302833.195814:0:6696:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0000: cli 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0/ffff960f10086800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113a27 ocd_grant: 134217728 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789302833.195832:0:6696:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0000: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789302833.195984:0:6696:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0000: get connection from MDS 0 00000100:00080000:2.0:1789302833.196015:0:6696:0:(import.c:65:import_set_state_nolock()) ffff960f10084800 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789302833.196053:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2344:x1876219350443136:12345-192.168.203.11@tcp:8:mount.lustre.0 Request processed in 330us (1060us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.196061:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 348 00000100:00100000:2.0:1789302833.196077:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 349 00000100:00100000:2.0:1789302833.196082:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:0+-99:2344:x1876219350443264:12345-192.168.203.11@tcp:8:mount.lustre.0 00010000:00080000:2.0:1789302833.196098:0:6696:0:(ldlm_lib.c:1453:target_handle_connect()) lustre-OST0001: connection from 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0@192.168.203.11@tcp t0 exp 0000000000000000 cur 670 last 0 00000020:00000080:2.0:1789302833.196115:0:6696:0:(genops.c:1384:class_connect()) connect: client 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0, cookie 0x64fe43095184eb3e 00002000:00100000:2.0:1789302833.196120:0:6696:0:(ofd_obd.c:121:ofd_parse_connect_data()) lustre-OST0001: cli 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0/ffff960f10082800 ocd_connect_flags: 0xa0467af2e3640038 ocd_version: 2113a27 ocd_grant: 134217728 ocd_index: 0 ocd_group 0 00002000:00100000:2.0:1789302833.196126:0:6696:0:(ofd_obd.c:219:ofd_parse_connect_data()) lustre-OST0001: cli supports cksum type f7, return f7 00002000:00080000:2.0:1789302833.196201:0:6696:0:(ofd_obd.c:385:ofd_obd_connect()) lustre-OST0001: get connection from MDS 0 00000100:00080000:2.0:1789302833.196216:0:6696:0:(import.c:65:import_set_state_nolock()) ffff960f1d83c000 : changing import state from RECOVER to FULL 00000100:00100000:2.0:1789302833.196238:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2344:x1876219350443264:12345-192.168.203.11@tcp:8:mount.lustre.0 Request processed in 155us (1217us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.196245:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 349 00000100:00100000:0.0:1789302833.563131:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.563266:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350443520 00000100:00100000:2.0:1789302833.563282:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350443520 00000100:00100000:2.0:1789302833.563359:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1263 00000100:00100000:2.0:1789302833.563371:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30758:x1876219350443520:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302833.563400:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302833.563429:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eb45 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a7a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.563757:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.563777:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.563796:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302833.563858:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302833.563864:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302833.563891:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.563992:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eb45 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 8 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.564014:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 7 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.564025:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:2.0:1789302833.564029:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302833.564033:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:2.0:1789302833.564180:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30758:x1876219350443520:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 810us (1062us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.564189:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1263 00000100:00100000:3.0:1789302833.576089:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302833.576175:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350443648 00000100:00100000:0.0:1789302833.576189:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350443648 00000100:00100000:0.0:1789302833.576270:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1264 00000100:00100000:0.0:1789302833.576283:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30759:x1876219350443648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302833.576312:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302833.576340:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eb53 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a81 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576434:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576453:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576472:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576490:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576497:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576534:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576897:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eb68 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576908:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eb68 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576916:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eb68 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302833.576955:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302833.576959:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302833.576983:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eb68 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.576997:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577004:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577017:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577023:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302833.577028:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577036:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577051:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:0.0:1789302833.577054:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302833.577061:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577068:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184eb5a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577084:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577090:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577096:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577101:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302833.577104:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577114:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577122:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b979c7a (ffff960f09ae1100) 00010000:00010000:0.0:1789302833.577125:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302833.577128:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577133:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184eb61 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b979c7a bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577166:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184eb53 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a81 expref: 9 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577176:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eb68 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a81 expref: 8 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.577182:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:0.0:1789302833.577183:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302833.577185:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044600, rc 0) 00000100:00100000:0.0:1789302833.577787:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:30759:x1876219350443648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1504us (1713us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302833.577797:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1264 00000100:00100000:1.0:1789302833.587843:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.593818:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350443904 00000100:00100000:2.0:1789302833.593850:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350443904 00000100:00100000:2.0:1789302833.594003:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1265 00000100:00100000:2.0:1789302833.594015:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:30759:x1876219350443904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302833.594042:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302833.594069:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eb6f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a88 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.594155:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.594172:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.594184:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.594201:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.594208:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.594216:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596587:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184eb84 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596618:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184eb84 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596634:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184eb84 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302833.596648:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00010000:00010000:2.0:1789302833.596798:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596807:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596815:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596832:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302833.596837:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596846:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596860:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:2.0:1789302833.596864:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302833.596870:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596876:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184eb76 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596891:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596898:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596903:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596908:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302833.596911:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596918:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596926:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff960f10d3c200) 00010000:00010000:2.0:1789302833.596929:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302833.596932:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.596938:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184eb7d lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.599961:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eb6f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a88 expref: 11 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.600001:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184eb84 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a88 expref: 10 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.600011:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302833.600015:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302833.600019:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:2.0:1789302833.600200:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30759:x1876219350443904:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 6184us (12380us total) trans 4294968143 rc 0/0 00000100:00100000:2.0:1789302833.600212:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1265 00000100:00100000:3.0:1789302833.602679:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.604108:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350444032 00000100:00100000:2.0:1789302833.604127:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350444032 00000100:00100000:3.0:1789302833.604311:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1266 00000100:00100000:3.0:1789302833.604326:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30759:x1876219350444032:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1789302833.604524:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30759:x1876219350444032:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 176us (1844us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.604533:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1266 00000040:00080000:0.0:1789302833.612802:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302833.612817:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:3.0:1789302833.613128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.613452:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350444160 00000100:00100000:1.0:1789302833.613466:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350444160 00000100:00100000:1.0:1789302833.613659:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1267 00000100:00100000:1.0:1789302833.613680:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30760:x1876219350444160:12345-192.168.203.11@tcp:101:lfs.0 00000040:00080000:0.0:1789302833.613695:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1789302833.613706:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302833.613707:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302833.613735:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eb92 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a8f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.613840:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.613859:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.613874:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.613892:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.613900:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.613907:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614156:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614165:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614173:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614179:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.614185:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614199:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614213:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302833.614216:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.614221:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614228:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184eb99 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614240:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614246:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614252:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614258:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.614260:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614267:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614273:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff960f2c94db00) 00010000:00010000:1.0:1789302833.614275:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.614278:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614287:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184eba0 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614325:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eb92 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a8f expref: 98 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614349:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302833.614352:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.614356:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eb92 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a8f expref: 98 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.614367:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:1.0:1789302833.614454:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30760:x1876219350444160:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 777us (1333us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302833.614463:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1267 00000100:00100000:3.0:1789302833.619556:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.619645:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350444416 00000100:00100000:2.0:1789302833.619659:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350444416 00000100:00100000:2.0:1789302833.619745:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1268 00000100:00100000:2.0:1789302833.619757:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30760:x1876219350444416:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:2.0:1789302833.619866:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302833.619909:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184e70f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08c8 expref: 97 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.619934:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:2.0:1789302833.619941:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302833.620246:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.620264:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 44 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.620278:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.620286:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 5/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.620329:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 9 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.620347:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 9 pid: 5844 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302833.620365:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371153152:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302833.620538:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 44 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.620550:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 44 pid: 5843 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302833.620559:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371153280:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302833.620596:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc663c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302833.630732:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.630781:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.631081:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350444544 00000100:00100000:3.0:1789302833.631094:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350444544 00010000:00010000:3.0:1789302833.631146:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 10 pid: 5844 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302833.631196:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350444672 00000100:00100000:3.0:1789302833.631199:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350444672 00010000:00010000:3.0:1789302833.631215:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 45 pid: 5843 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302833.631241:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 379 00000100:00100000:3.0:1789302833.631252:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2348:x1876219350444544:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302833.631271:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302833.631279:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 10 pid: 5844 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631294:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 10 pid: 5844 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631322:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302833.631328:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631340:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302833.631410:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2348:x1876219350444544:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 159us (693us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.631418:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 379 00000100:00100000:3.0:1789302833.631443:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 380 00000100:00100000:3.0:1789302833.631450:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2348:x1876219350444672:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302833.631458:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302833.631464:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 45 pid: 5843 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631475:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 45 pid: 5843 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631487:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302833.631491:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631533:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631544:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631569:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302833.631606:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:2348:x1876219350444672:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 156us (824us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.631613:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 380 00010000:00010000:3.0:1789302833.631676:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184eb4c lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a7a expref: 9 pid: 5844 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302833.631693:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371153152:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302833.631707:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184e74e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a08dd expref: 44 pid: 5843 timeout: 770 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302833.631719:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371153280:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302833.631747:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631758:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631764:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631793:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631803:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.631811:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302833.633108:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 16384 00010000:00010000:3.0:1789302833.633128:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.633138:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.633145:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.633151:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302833.633156:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.633165:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.633186:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302833.633189:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302833.633195:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.633202:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184eba7 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302833.633224:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000001ff0093d reply state 00000000174d9b21 transno 4294968144 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302833.633234:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302833.633350:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30760:x1876219350444416:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 13591us (13804us total) trans 4294968144 rc 0/0 00000100:00100000:3.0:1789302833.633359:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1268 00010000:00010000:3.0:1789302833.641756:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.641779:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.641787:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.641794:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302833.641799:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.641807:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.641846:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302833.641850:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302833.641857:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.641864:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebae lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302833.660381:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.660763:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350444800 00000100:00100000:1.0:1789302833.660779:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350444800 00000100:00100000:1.0:1789302833.660913:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1269 00000100:00100000:1.0:1789302833.660925:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:30764:x1876219350444800:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:1.0:1789302833.660956:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302833.660993:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ebb5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a96 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.661088:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebbc lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.661106:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebbc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.661121:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebbc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302833.661177:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302833.661181:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302833.661203:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebbc lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.661231:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ebb5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a96 expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.661251:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebbc lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a96 expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.661260:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:1.0:1789302833.661263:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302833.661267:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7c00, rc 0) 00000100:00100000:1.0:1789302833.661355:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30764:x1876219350444800:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 433us (991us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302833.661362:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1269 00000100:00100000:1.0:1789302833.665189:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.665242:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.665321:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350444928 00000100:00100000:2.0:1789302833.665334:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350444928 00000100:00100000:2.0:1789302833.665401:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350445184 00000100:00100000:2.0:1789302833.665405:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350445184 00000100:00100000:2.0:1789302833.665436:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1270 00000100:00100000:2.0:1789302833.665447:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350444928:12345-192.168.203.11@tcp:41:mkdir.0 00000004:00000024:2.0:1789302833.665521:0:5843:0:(mdt_handler.c:540:mdt_statfs()) blocks cached 0 granted 18087936 pending 0 free 2259681280 avail 2257584128 00000100:00100000:2.0:1789302833.665607:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350444928:12345-192.168.203.11@tcp:41:mkdir.0 Request processed in 161us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302833.665615:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1270 00000100:00100000:2.0:1789302833.665637:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1271 00000100:00100000:2.0:1789302833.665644:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30764:x1876219350445184:12345-192.168.203.11@tcp:36:mkdir.0 00010000:00010000:2.0:1789302833.665693:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302833.665726:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebbc lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a96 expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302833.665750:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:2.0:1789302833.665764:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302833.666040:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:30764:x1876219350445184:12345-192.168.203.11@tcp:36:mkdir.0 Request processed in 395us (796us total) trans 0 rc -17/-17 00000100:00100000:2.0:1789302833.666048:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1271 00000100:00100000:2.0:1789302833.669378:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.669475:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350445312 00000100:00100000:1.0:1789302833.669489:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350445312 00000100:00100000:1.0:1789302833.669597:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1272 00000100:00100000:1.0:1789302833.669610:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+44:30764:x1876219350445312:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:1.0:1789302833.669637:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302833.669668:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ebc3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0a9d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.669761:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.669779:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.669794:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302833.669852:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302833.669857:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302833.669881:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.669924:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ebc3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.669946:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 45 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.669955:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:1.0:1789302833.669959:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302833.669963:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:1.0:1789302833.670048:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30764:x1876219350445312:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 439us (683us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302833.670122:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1272 00000100:00100000:2.0:1789302833.672810:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.673057:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350445440 00000100:00100000:3.0:1789302833.673080:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350445440 00000100:00100000:3.0:1789302833.673149:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1273 00000100:00100000:3.0:1789302833.673161:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+45:30764:x1876219350445440:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302833.673187:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302833.673211:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebd1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0aa4 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673299:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673315:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673328:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673345:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673352:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673360:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673482:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ebe6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673491:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ebe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673498:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ebe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302833.673572:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 16384 00000004:00200000:3.0:1789302833.673577:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302833.673599:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ebe6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673615:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673621:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673629:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673636:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302833.673641:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673649:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673664:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:3.0:1789302833.673667:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302833.673673:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673680:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ebd8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673694:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673700:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673706:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673712:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302833.673715:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673721:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673730:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302833.673732:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302833.673735:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673741:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ebdf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673797:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ebd1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aa4 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673814:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ebe6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aa4 expref: 46 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.673832:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:3.0:1789302833.673835:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302833.673837:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302833.674370:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30764:x1876219350445440:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 1206us (1575us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.674382:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1273 00000100:00100000:3.0:1789302833.686621:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.687014:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350445568 00000100:00100000:1.0:1789302833.687028:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350445568 00000100:00100000:1.0:1789302833.687094:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1274 00000100:00100000:1.0:1789302833.687105:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30765:x1876219350445568:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302833.687132:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302833.687158:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ebed lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0aab expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687240:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687327:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687342:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687361:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687369:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687375:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687449:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687462:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687469:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687603:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.687608:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687616:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687630:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302833.687633:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.687638:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687646:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ebf4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687660:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687665:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687671:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687677:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.687680:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687686:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687692:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f2c94db00) 00010000:00010000:1.0:1789302833.687694:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.687696:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687701:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ebfb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687739:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ebed lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aab expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687749:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302833.687751:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.687754:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ebed lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aab expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.687763:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:1.0:1789302833.687852:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30765:x1876219350445568:12345-192.168.203.11@tcp:101:rm.0 Request processed in 748us (1244us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302833.687860:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1274 00000100:00100000:2.0:1789302833.707028:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.707147:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350445824 00000100:00100000:1.0:1789302833.707162:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350445824 00000100:00100000:1.0:1789302833.707242:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1275 00000100:00100000:1.0:1789302833.707269:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30766:x1876219350445824:12345-192.168.203.11@tcp:101:dd.0 00010000:00010000:1.0:1789302833.707301:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302833.707305:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789302833.707343:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ebe6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aa4 expref: 46 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.707370:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f0cdf2900) 00010000:00010000:1.0:1789302833.707391:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302833.707412:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec02 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ab2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.707638:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.707655:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.707668:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.707682:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.707689:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.707695:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302833.708464:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1a:0x0], index 1: 0 00010000:00010000:1.0:1789302833.708730:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708746:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708758:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708800:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708808:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708815:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708834:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.708840:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708849:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708862:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:1.0:1789302833.708865:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.708870:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708877:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec09 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302833.708890:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000554c54fb reply state 00000000c00d0ece transno 4294968145 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302833.708901:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302833.708941:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec02 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708962:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.708971:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:1.0:1789302833.708973:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302833.708976:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6a00, rc 0) 00000100:00100000:1.0:1789302833.709064:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30766:x1876219350445824:12345-192.168.203.11@tcp:101:dd.0 Request processed in 1810us (2052us total) trans 4294968145 rc 0/0 00000100:00100000:1.0:1789302833.709075:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1275 00010000:00010000:1.0:1789302833.720039:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.720063:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.720074:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.720081:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.720087:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.720097:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.720124:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f2c94d900) 00010000:00010000:1.0:1789302833.720127:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.720133:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.720140:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec10 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302833.739423:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.739686:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350445952 00000100:00100000:3.0:1789302833.739702:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350445952 00000100:00100000:3.0:1789302833.739789:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 350 00000100:00100000:3.0:1789302833.739803:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:30766:x1876219350445952:12345-192.168.203.11@tcp:101:dd.0 00010000:00010000:3.0:1789302833.739836:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302833.739865:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ab9 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302833.739907:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1a:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302833.739936:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=670s v=(0 0) 00010000:00010000:3.0:1789302833.739941:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302833.739952:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.739984:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302833.740000:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302833.740390:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:30766:x1876219350445952:12345-192.168.203.11@tcp:101:dd.0 Request processed in 586us (982us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.740399:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 350 00000100:00100000:3.0:1789302833.755788:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302833.755971:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350446080 00000100:00100000:2.0:1789302833.755984:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350446080 00002000:00100000:2.0:1789302833.756051:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1a:0x0] (0->1048575) req@ffff960f3ef7ad80 x1876219350446080/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:409/0 lens 488/0 e 0 to 0 dl 1789302844 ref 1 fl New:/600/ffffffff rc 0/-1 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302833.756064:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7ad80 with x1876219350446080 ext(0->1048575) 00002000:00010000:2.0:1789302833.756071:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ef7ad80 00000100:00100000:2.0:1789302833.756113:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 113 00000100:00100000:2.0:1789302833.756124:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350446080:12345-192.168.203.11@tcp:4:dd.0 00000100:00100000:1.0:1789302833.780255:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350446080:12345-192.168.203.11@tcp:4:dd.0 Request processed in 24117us (24467us total) trans 4294967371 rc 0/0 00000100:00100000:1.0:1789302833.780279:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 113 00002000:00100000:1.0:1789302833.780295:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1a:0x0] (0->1048575) req@ffff960f3ef7ad80 x1876219350446080/t4294967371(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:409/0 lens 488/448 e 0 to 0 dl 1789302844 ref 1 fl Complete:/600/0 rc 0/0 job:'dd.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302833.780371:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7ad80 with x1876219350446080 ext(0->1048575) 00002000:00010000:1.0:1789302833.780381:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ef7ad80 00000100:00100000:0.0:1789302833.783473:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.783804:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350446208 00000100:00100000:3.0:1789302833.783828:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350446208 00000100:00100000:3.0:1789302833.784008:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1276 00000100:00100000:3.0:1789302833.784025:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30766:x1876219350446208:12345-192.168.203.11@tcp:44:dd.0 00010000:00010000:1.0:1789302833.805713:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302833.871694:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968145 is committed 00000001:00080000:1.0:1789302833.871720:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968143 is committed 00000001:00100000:1.0:1789302833.871727:0:5865:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000001:00080000:1.0:1789302833.871734:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968144 is committed 00000020:00080000:1.0:1789302833.871743:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 28 batchid updated 00000020:00080000:0.0:1789302833.872030:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 28 00000020:00080000:0.0:1789302833.872045:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 28 00000100:00100000:3.0:1789302833.872124:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30766:x1876219350446208:12345-192.168.203.11@tcp:44:dd.0 Request processed in 88097us (88658us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.872137:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1276 00000100:00100000:2.0:1789302833.876043:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.877155:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350446336 00000100:00100000:3.0:1789302833.877173:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350446336 00000100:00100000:3.0:1789302833.877253:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 351 00000100:00100000:3.0:1789302833.877267:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350446336:12345-192.168.203.11@tcp:16:dd.0 00000100:00100000:3.0:1789302833.922326:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350446336:12345-192.168.203.11@tcp:16:dd.0 Request processed in 45042us (46279us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302833.922342:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 351 00000001:00080000:2.0:1789302833.922953:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967371 is committed 00000001:00100000:2.0:1789302833.922978:0:7654:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000100:00100000:2.0:1789302833.927863:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302833.927971:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350446592 00000100:00100000:3.0:1789302833.927984:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350446592 00000100:00100000:3.0:1789302833.928057:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 258 00000100:00100000:3.0:1789302833.928070:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30766:x1876219350446592:12345-192.168.203.11@tcp:35:dd.0 00000100:00100000:2.0:1789302833.929723:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30766:x1876219350446592:12345-192.168.203.11@tcp:35:dd.0 Request processed in 1645us (1862us total) trans 4294968146 rc 0/0 00000100:00100000:2.0:1789302833.929739:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 258 00000100:00100000:0.0:1789302833.950716:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302833.950815:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350446720 00000100:00100000:1.0:1789302833.950842:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350446720 00000100:00100000:1.0:1789302833.950921:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1277 00000100:00100000:1.0:1789302833.950933:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30767:x1876219350446720:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302833.950967:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302833.950996:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec2c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ac0 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951163:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951182:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951196:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951214:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951222:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951229:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951645:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184ec41 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951656:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184ec41 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951664:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184ec41 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302833.951699:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 16384 00000004:00200000:1.0:1789302833.951702:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302833.951723:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184ec41 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951738:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951745:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951753:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951759:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.951766:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951775:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951788:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:1.0:1789302833.951792:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.951798:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951805:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184ec33 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951830:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951837:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951843:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951850:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302833.951853:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951859:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951867:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff960f2c94d900) 00010000:00010000:1.0:1789302833.951869:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302833.951872:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951879:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec3a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951916:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec2c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ac0 expref: 49 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951931:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662800/0x64fe43095184ec41 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ac0 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302833.951939:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:1.0:1789302833.951942:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302833.951944:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01662800, rc 0) 00000100:00100000:1.0:1789302833.952171:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:30767:x1876219350446720:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1237us (1468us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302833.952180:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1277 00000100:00100000:3.0:1789302833.998327:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302833.998457:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350446848 00000100:00100000:0.0:1789302833.998472:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350446848 00000100:00100000:0.0:1789302833.998597:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1278 00000100:00100000:0.0:1789302833.998610:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+96:30769:x1876219350446848:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:0.0:1789302833.998645:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302833.998676:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ec48 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ac7 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.998768:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ec4f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.998786:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ec4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.998803:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ec4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302833.998861:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302833.999200:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302833.999263:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ec4f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.999296:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ec48 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ac7 expref: 98 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.999320:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ec4f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ac7 expref: 97 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302833.999335:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:0.0:1789302833.999338:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302833.999343:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045a00, rc 0) 00000100:00100000:0.0:1789302833.999425:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30769:x1876219350446848:12345-192.168.203.11@tcp:101:stat.0 Request processed in 818us (1115us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302833.999434:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1278 00000100:00100000:3.0:1789302834.008701:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302834.009240:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350446976 00000100:00100000:2.0:1789302834.009255:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350446976 00000100:00100000:2.0:1789302834.009352:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1279 00000100:00100000:2.0:1789302834.009364:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+97:30769:x1876219350446976:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:2.0:1789302834.009402:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302834.009430:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ec56 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ace expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.009577:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.009595:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.009610:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.009626:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.009635:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.009642:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010196:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ec6b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010208:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ec6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010216:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ec6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302834.010251:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 16384 00000004:00200000:2.0:1789302834.010256:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302834.010282:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ec6b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010297:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010303:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010310:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010317:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302834.010323:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010333:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010346:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:2.0:1789302834.010349:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302834.010355:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010362:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ec5d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010375:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010381:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010388:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010392:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302834.010394:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010399:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010405:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff960f10d3c200) 00010000:00010000:2.0:1789302834.010408:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302834.010410:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010420:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ec64 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010457:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ec56 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ace expref: 99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010471:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3600/0x64fe43095184ec6b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ace expref: 98 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.010478:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:2.0:1789302834.010481:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302834.010483:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3600, rc 0) 00000100:00100000:2.0:1789302834.010602:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30769:x1876219350446976:12345-192.168.203.11@tcp:101:stat.0 Request processed in 1240us (1915us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302834.010611:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1279 00000100:00080000:0.0:1789302834.012172:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302834.016024:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302834.016201:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350447104 00000100:00100000:1.0:1789302834.016216:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350447104 00000100:00100000:1.0:1789302834.016290:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1280 00000100:00100000:1.0:1789302834.016302:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+98:30769:x1876219350447104:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:1.0:1789302834.016333:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302834.016364:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184ec72 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ad5 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.016479:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.016679:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.016698:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.016722:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.016731:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.016738:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017290:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017302:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017310:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302834.017357:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:1.0:1789302834.017362:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302834.017402:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017417:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017426:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017433:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017440:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017447:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302834.017452:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.017461:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018148:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:1.0:1789302834.018156:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302834.018164:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018178:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095184ec79 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018207:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018214:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018221:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018227:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302834.018231:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018238:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018246:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f2c94d900) 00010000:00010000:1.0:1789302834.018249:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302834.018252:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018259:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec80 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018319:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe43095184ec72 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 100 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018337:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.018351:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:1.0:1789302834.018354:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302834.018357:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7c00, rc 0) 00000100:00100000:1.0:1789302834.018755:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30769:x1876219350447104:12345-192.168.203.11@tcp:101:stat.0 Request processed in 2452us (2983us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302834.018765:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1280 00000100:00100000:3.0:1789302834.022400:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.022479:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350447232 00000100:00100000:0.0:1789302834.022494:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350447232 00000100:00100000:0.0:1789302834.024033:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 352 00000100:00100000:0.0:1789302834.024057:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350447232:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:0.0:1789302834.024085:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302834.024115:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ec8e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0adc expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302834.024212:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b08700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371153408:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302834.024363:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4d80 going to sleep for 11 seconds 00000100:00080000:1.0:1789302834.024429:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:0.0:1789302834.031836:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb size: 0 -> 1048576 00002000:00010000:0.0:1789302834.031848:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb mtime: 0 -> 1789302833 00002000:00010000:0.0:1789302834.031851:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb atime: 0 -> 1789302833 00002000:00010000:0.0:1789302834.031854:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb ctime: 0 -> 1789302833 00002000:00010000:0.0:1789302834.031856:0:6695:0:(ofd_lvb.c:284:ofd_lvbo_update()) res: [0x280000400:0x1a:0x0] updating lvb blocks: 1 -> 2051 00000100:00100000:0.0:1789302834.031903:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b08700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371153408:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302834.031962:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ec8e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0adc expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.032001:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ec8e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0adc expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.032018:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044400, rc 0) 00000100:00100000:0.0:1789302834.032105:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350447232:12345-192.168.203.11@tcp:101:stat.0 Request processed in 8056us (9718us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302834.032116:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 352 00000100:00100000:3.0:1789302834.055340:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302834.055610:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350447360 00000100:00100000:2.0:1789302834.055625:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350447360 00000100:00100000:2.0:1789302834.055702:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 353 00000100:00100000:2.0:1789302834.055715:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350447360:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302834.055741:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302834.055768:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ec95 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ae3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302834.055994:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371153536:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302834.056107:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc666c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302834.058794:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371153536:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302834.058854:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ec95 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ae3 expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.058891:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ec95 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ae3 expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.058913:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302834.058981:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350447360:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3268us (3659us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302834.058991:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 353 00000100:00100000:3.0:1789302834.065689:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302834.065909:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350447488 00000100:00100000:1.0:1789302834.065923:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350447488 00000100:00100000:1.0:1789302834.065992:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1281 00000100:00100000:1.0:1789302834.066004:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30770:x1876219350447488:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302834.066032:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302834.066059:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec9c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0aea expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.066154:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.066172:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.066186:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302834.066232:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 16384 00000004:00200000:1.0:1789302834.066236:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302834.066418:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.066456:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe43095184ec9c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 101 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.066479:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 100 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.066489:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:1.0:1789302834.066493:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302834.066497:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:1.0:1789302834.066705:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30770:x1876219350447488:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 702us (1030us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302834.066712:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1281 00000100:00100000:3.0:1789302834.071463:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302834.071727:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350447616 00000100:00100000:2.0:1789302834.071741:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350447616 00000100:00100000:2.0:1789302834.071808:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 354 00000100:00100000:2.0:1789302834.071829:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350447616:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302834.071853:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302834.071886:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ecaa lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0af1 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302834.071972:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371153664:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302834.072094:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc666c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302834.076392:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371153664:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302834.076456:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ecaa lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af1 expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.076654:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe43095184ecaa lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af1 expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.076674:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302834.076798:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350447616:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4979us (5348us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302834.076808:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 354 00000100:00100000:0.0:1789302834.084249:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302834.084543:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350447872 00000100:00100000:1.0:1789302834.084556:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350447872 00000100:00100000:1.0:1789302834.084623:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1282 00000100:00100000:1.0:1789302834.084635:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30770:x1876219350447872:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:1.0:1789302834.085451:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].37, 136 off13224 00000100:00100000:1.0:1789302834.085717:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30770:x1876219350447872:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 1078us (1479us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302834.085728:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1282 00000040:00080000:0.0:1789302834.636745:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302834.636755:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302834.636800:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 38 count 5 00000040:00080000:0.0:1789302834.636808:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302834.637456:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbb480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371153792:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302834.638959:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4300 going to sleep for 11 seconds 00000100:00100000:0.0:1789302834.645007:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302834.645095:0:5855:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350448000 00000100:00100000:3.0:1789302834.645106:0:5855:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350448000 00000100:00100000:1.0:1789302834.645155:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbb480 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371153792:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302834.645196:0:5855:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 3 00000100:00100000:3.0:1789302834.645211:0:5855:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30780:x1876219350448000:12345-192.168.203.11@tcp:900:lhsmtool_posix.0 00000100:00100000:3.0:1789302834.645329:0:5855:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_fld_0001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30780:x1876219350448000:12345-192.168.203.11@tcp:900:lhsmtool_posix.0 Request processed in 121us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302834.645340:0:5855:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 3 00000100:00100000:3.0:1789302834.648484:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302834.648588:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350448128 00000100:00100000:1.0:1789302834.648604:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350448128 00000100:00100000:1.0:1789302834.648682:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1283 00000100:00100000:1.0:1789302834.648692:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30780:x1876219350448128:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302834.648741:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302834.648857:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30780:x1876219350448128:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 166us (398us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302834.648864:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1283 00000100:00100000:2.0:1789302834.661460:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302834.661762:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350448384 00000100:00100000:3.0:1789302834.661779:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350448384 00000100:00100000:3.0:1789302834.661871:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1284 00000100:00100000:3.0:1789302834.661896:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30780:x1876219350448384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302834.661929:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302834.661960:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ecb1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0af8 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302834.662578:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302834.662598:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302834.662613:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302834.662629:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:3.0:1789302834.663959:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ecb1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 10 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302834.664186:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 9 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302834.664196:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:3.0:1789302834.664199:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302834.664203:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302834.664307:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30780:x1876219350448384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2423us (2866us total) trans 4294968147 rc 0/0 00000100:00100000:3.0:1789302834.664318:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1284 00000100:00100000:3.0:1789302834.670178:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302834.670388:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350448512 00000100:00100000:1.0:1789302834.670404:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350448512 00000100:00080000:0.0:1789302834.670706:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302834.670786:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 355 00000100:00100000:1.0:1789302834.670819:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2347:x1876219350448512:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302834.670894:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.670923:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 9 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.670934:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1a:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302834.670939:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302834.670944:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.671002:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 9 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.671019:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 9 pid: 18578 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302834.671037:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371153920:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302834.671159:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e5993c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302834.675320:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095afb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371153920:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302834.675367:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302834.681494:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.681665:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350448640 00000100:00100000:0.0:1789302834.681680:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350448640 00010000:00010000:0.0:1789302834.681768:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 10 pid: 18578 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302834.681808:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 381 00000100:00100000:0.0:1789302834.681829:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350448640:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302834.681843:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302834.681867:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 10 pid: 18578 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.681882:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 10 pid: 18578 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.681907:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aedbc85e/0x64fe43095184ec25 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab9 expref: 10 pid: 18578 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.681926:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302834.681930:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302834.681936:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.681947:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.681995:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302834.682060:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350448640:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 232us (582us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302834.682071:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 381 00010000:00010000:2.0:1789302834.682355:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682367:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682393:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682402:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682410:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682416:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302834.682421:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682437:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682450:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.682457:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe43095184ecc6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302834.682583:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2347:x1876219350448512:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 11785us (12419us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302834.682592:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 355 00000100:00080000:1.0:1789302834.682956:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302834.693272:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.696139:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350448896 00000100:00100000:0.0:1789302834.696157:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350448896 00000100:00100000:0.0:1789302834.696240:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1285 00000100:00100000:0.0:1789302834.696254:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30780:x1876219350448896:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302834.696384:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30780:x1876219350448896:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 133us (3125us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302834.696397:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1285 00000100:00100000:1.0:1789302834.707978:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302834.709766:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350449024 00000100:00100000:3.0:1789302834.709780:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350449024 00000100:00100000:3.0:1789302834.709869:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 356 00000100:00100000:3.0:1789302834.709880:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350449024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302834.709899:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302834.709923:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0aff expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302834.709953:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1a:0x0] initial LVB size: 1048576, mtime: 0x6aa69831, atime: 0x0, ctime: 0x6aa69831, blocks: 0x803 00010000:00010000:3.0:1789302834.709974:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=671s v=(0 0) 00010000:00010000:3.0:1789302834.709979:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302834.709988:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 6 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302834.710018:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 6 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302834.710032:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:3.0:1789302834.710098:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2348:x1876219350449024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 220us (2129us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302834.710105:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 356 00000100:00100000:3.0:1789302834.716095:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.716281:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350449280 00000100:00100000:0.0:1789302834.716291:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350449280 00000100:00100000:0.0:1789302834.716348:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1286 00000100:00100000:0.0:1789302834.716359:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30780:x1876219350449280:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302834.718264:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30780:x1876219350449280:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 1896us (2168us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302834.718277:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1286 00000100:00100000:1.0:1789302834.735360:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302834.735630:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350449408 00000100:00100000:3.0:1789302834.735640:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350449408 00002000:00100000:3.0:1789302834.735693:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1a:0x0] (0->1048575) req@ffff960f3f709f80 x1876219350449408/t0(0) o3->36d456e5-2a1b-4d58-a7f1-78ab3c35acb0@192.168.203.11@tcp:410/0 lens 488/0 e 0 to 0 dl 1789302845 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302834.735704:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f709f80 with x1876219350449408 ext(0->1048575) 00002000:00010000:3.0:1789302834.735710:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f709f80 00000100:00100000:3.0:1789302834.735735:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 114 00000100:00100000:3.0:1789302834.735745:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2346:x1876219350449408:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302834.765745:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2346:x1876219350449408:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 29991us (30383us total) trans 0 rc 1048576/1048576 00000100:00100000:3.0:1789302834.765758:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 114 00002000:00100000:3.0:1789302834.765770:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1a:0x0] (0->1048575) req@ffff960f3f709f80 x1876219350449408/t0(0) o3->36d456e5-2a1b-4d58-a7f1-78ab3c35acb0@192.168.203.11@tcp:410/0 lens 488/440 e 0 to 0 dl 1789302845 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302834.765780:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f709f80 with x1876219350449408 ext(0->1048575) 00002000:00010000:3.0:1789302834.765786:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f709f80 00000100:00100000:1.0:1789302834.810636:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302834.810735:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350449536 00000100:00100000:2.0:1789302834.810746:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350449536 00000100:00100000:2.0:1789302834.810810:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1287 00000100:00100000:2.0:1789302834.810830:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30780:x1876219350449536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302834.810850:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302834.810872:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ecd4 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b06 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.810916:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.810930:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.810944:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.811035:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ecd4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 11 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.811054:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 10 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.811062:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:2.0:1789302834.811065:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302834.811068:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302834.811126:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30780:x1876219350449536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 298us (499us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302834.811132:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1287 00000100:00100000:1.0:1789302834.815248:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302834.815327:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350449664 00000100:00100000:3.0:1789302834.815338:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350449664 00000100:00100000:3.0:1789302834.815400:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1288 00000100:00100000:3.0:1789302834.815409:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30780:x1876219350449664:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302834.815439:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302834.815573:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30780:x1876219350449664:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 164us (334us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302834.815580:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1288 00000100:00100000:2.0:1789302834.820771:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.820842:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350449792 00000100:00100000:0.0:1789302834.820852:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350449792 00000100:00100000:0.0:1789302834.820992:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:0.0:1789302834.821002:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2347:x1876219350449792:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302834.821045:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821063:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302834.821067:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302834.821073:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821088:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821119:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821132:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821141:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821148:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302834.821153:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821162:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821177:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.821184:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000aa8d8456/0x64fe43095184ece2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302834.821266:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2347:x1876219350449792:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 258us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302834.821273:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 357 00000100:00100000:1.0:1789302834.828871:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302834.829780:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302834.830769:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350450048 00000100:00100000:0.0:1789302834.830781:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350450048 00000100:00100000:0.0:1789302834.830852:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1289 00000100:00100000:0.0:1789302834.830863:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30780:x1876219350450048:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302834.831033:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.831051:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 10 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.831060:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.831095:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 10 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.831111:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 10 pid: 6919 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302834.831125:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbb800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371154048:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302834.831245:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded43c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302834.837931:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.838424:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbb800 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371154048:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302834.838456:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302834.838592:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350450176 00000100:00100000:2.0:1789302834.838599:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350450176 00010000:00010000:2.0:1789302834.838635:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 11 pid: 6919 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302834.838686:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 382 00000100:00100000:2.0:1789302834.838697:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:2348:x1876219350450176:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302834.838712:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302834.838718:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 11 pid: 6919 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.838737:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 11 pid: 6919 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.838764:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ecdb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b06 expref: 11 pid: 6919 timeout: 771 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.838776:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:2.0:1789302834.838781:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.838790:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.838798:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302834.838836:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302834.838957:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.838965:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302834.839045:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2348:x1876219350450176:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 349us (1121us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302834.839053:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 382 00010000:00010000:1.0:1789302834.839149:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.839155:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.839159:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.839163:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302834.839166:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.839172:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.839179:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:1.0:1789302834.839181:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302834.839183:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.839187:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ece9 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789302834.839496:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302834.839522:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302834.840111:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 5 00000040:00080000:2.0:1789302834.840120:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302834.840832:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30780:x1876219350450048:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 9965us (11965us total) trans 4294968148 rc 0/0 00000100:00100000:1.0:1789302834.840845:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1289 00000100:00100000:1.0:1789302834.842852:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.843029:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350450432 00000100:00100000:0.0:1789302834.843039:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350450432 00000100:00100000:0.0:1789302834.843089:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 259 00000100:00100000:0.0:1789302834.843098:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30780:x1876219350450432:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302834.843328:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30780:x1876219350450432:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 229us (482us total) trans 4294968149 rc 0/0 00000100:00100000:0.0:1789302834.843336:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 259 00000040:00080000:3.0:1789302834.893957:0:21511:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302834.893965:0:21511:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302834.894016:0:21511:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 38 count 5 00000040:00080000:3.0:1789302834.894021:0:21511:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302834.979242:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302834.979323:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350450560 00000100:00100000:0.0:1789302834.979333:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350450560 00000100:00100000:0.0:1789302834.979400:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 358 00000100:00100000:0.0:1789302834.979411:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350450560:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302834.979429:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302834.979448:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b0d expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.979475:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302834.979480:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302834.979489:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.979850:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302834.979865:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044400, rc 0) 00000100:00100000:0.0:1789302834.982827:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2345:x1876219350450560:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3397us (3574us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302834.982840:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 358 00000100:00100000:3.0:1789302834.994074:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302834.994221:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350450816 00000100:00100000:1.0:1789302834.994233:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350450816 00000100:00100000:1.0:1789302834.994294:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1290 00000100:00100000:1.0:1789302834.994305:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30781:x1876219350450816:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302834.994326:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302834.994349:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184ecf7 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b14 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.994451:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184ecfe lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.994466:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184ecfe lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.994479:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184ecfe lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302834.994491:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:1.0:1789302834.994660:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184ecf7 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b14 expref: 103 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.994679:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184ecfe lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b14 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302834.994687:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:1.0:1789302834.994690:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1789302834.994695:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9000, rc 0) 00000100:00100000:1.0:1789302834.994755:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30781:x1876219350450816:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 452us (691us total) trans 4294968150 rc 0/0 00000100:00100000:1.0:1789302834.994762:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1290 00000100:00100000:2.0:1789302835.011707:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302835.011769:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350450944 00000100:00100000:3.0:1789302835.011778:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350450944 00000100:00100000:3.0:1789302835.011849:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 359 00000100:00100000:3.0:1789302835.011859:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350450944:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:3.0:1789302835.012005:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.012037:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 5 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.012046:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.012054:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302835.012058:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302835.012062:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 5/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.012098:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.012112:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 18 pid: 6695 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302835.012126:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371154176:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302835.012231:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 5 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.012240:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 5 pid: 18578 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302835.012256:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371154304:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302835.012288:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302835.019472:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371154176:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302835.019488:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371154304:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302835.019655:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302835.020468:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302835.020545:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302835.020626:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350451072 00000100:00100000:0.0:1789302835.020636:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350451072 00010000:00010000:0.0:1789302835.020674:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 19 pid: 6695 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.020715:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350451200 00000100:00100000:0.0:1789302835.020718:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350451200 00010000:00010000:0.0:1789302835.020734:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 6 pid: 18578 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.020754:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00100000:0.0:1789302835.020763:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09751050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2347:x1876219350451072:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302835.020776:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302835.020796:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 19 pid: 6695 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.020808:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 19 pid: 6695 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.020843:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000628dfe70/0x64fe43095184ecf0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b0d expref: 19 pid: 6695 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.020861:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302835.020914:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09751050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350451072:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 150us (447us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302835.020922:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00100000:0.0:1789302835.020970:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 384 00000100:00100000:0.0:1789302835.020977:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2347:x1876219350451200:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302835.020987:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302835.021012:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 6 pid: 18578 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021023:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 6 pid: 18578 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021037:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184eccd lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aff expref: 6 pid: 18578 timeout: 772 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021048:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302835.021058:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302835.021065:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021074:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021100:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302835.021247:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2347:x1876219350451200:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 268us (700us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302835.021255:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 384 00010000:00010000:0.0:1789302835.021364:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021374:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021398:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021407:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021414:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021421:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302835.021424:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021432:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021446:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.021454:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed0c lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.021527:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350450944:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 9645us (9803us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302835.021535:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 359 00000100:00100000:3.0:1789302835.028537:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302835.029101:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350451456 00000100:00100000:0.0:1789302835.029699:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350451456 00000100:00100000:0.0:1789302835.029761:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 260 00000100:00100000:0.0:1789302835.029771:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076eb850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30781:x1876219350451456:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:0.0:1789302835.029847:0:5846:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x27:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184ecfe lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b14 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302835.031099:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968151 00000001:00080000:0.0:1789302835.035588:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968152 00010000:00010000:0.0:1789302835.035677:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035694:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035700:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035705:0:5846:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 10 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035710:0:5846:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 6/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035740:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 10 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035752:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 10 pid: 6919 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.035768:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbb800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371154432:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302835.035872:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035881:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 105 pid: 18427 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.035890:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fba680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371154560:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302835.035921:0:5846:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.035930:0:5846:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 47 pid: 5843 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.035939:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb9f80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371154688:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302835.035967:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4300 going to sleep for 11 seconds 00000100:00100000:1.0:1789302835.039478:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbb800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371154432:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302835.039491:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fba680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371154560:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302835.040560:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302835.040633:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350451584 00000100:00100000:0.0:1789302835.040642:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350451584 00010000:00010000:0.0:1789302835.040690:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 11 pid: 6919 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.040726:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 385 00000100:00100000:0.0:1789302835.040734:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:2347:x1876219350451584:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302835.040749:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302835.040756:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 11 pid: 6919 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.040768:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 11 pid: 6919 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.040788:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ecb8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0af8 expref: 11 pid: 6919 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.040802:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:0.0:1789302835.040807:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.040826:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.040832:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302835.040890:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2347:x1876219350451584:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 155us (335us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302835.040897:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 385 00000100:00080000:1.0:1789302835.040959:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302835.050716:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302835.050775:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350451840 00000100:00100000:2.0:1789302835.050785:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350451840 00010000:00010000:2.0:1789302835.050851:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 48 pid: 5843 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302835.050877:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb9f80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000_UUID:5846:1876219371154688:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302835.050887:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 386 00000100:00100000:2.0:1789302835.050895:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2346:x1876219350451840:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302835.050908:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302835.050909:0:5846:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.050915:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 48 pid: 5843 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.050925:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 48 pid: 5843 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.050947:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe43095184ec17 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ab2 expref: 48 pid: 5843 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.050961:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:2.0:1789302835.050965:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.050974:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.050979:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302835.051031:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2346:x1876219350451840:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 136us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302835.051038:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 386 00000100:00080000:0.0:1789302835.051102:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302835.053481:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302835.054185:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350451712 00000100:00100000:0.0:1789302835.054195:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350451712 00010000:00010000:0.0:1789302835.054230:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 106 pid: 18427 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302835.054353:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 387 00000100:00100000:0.0:1789302835.054363:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350451712:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302835.054378:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302835.054385:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 106 pid: 18427 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.054397:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 106 pid: 18427 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.054416:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095184ec87 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ad5 expref: 106 pid: 18427 timeout: 772 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.054429:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:0.0:1789302835.054434:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.054442:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.054450:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.054482:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302835.054549:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:2347:x1876219350451712:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 188us (1079us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302835.054556:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 387 00010000:00010000:2.0:1789302835.054615:0:5846:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.054623:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302835.054766:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968153 00000001:00080000:2.0:1789302835.054988:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968154 00000001:00080000:2.0:1789302835.055044:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968155 00000100:00100000:2.0:1789302835.055122:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371154816:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302835.055168:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302835.055195:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302835.055214:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371154816 00000100:00100000:2.0:1789302835.055218:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371154816 00000100:00100000:2.0:1789302835.055255:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 61 00000100:00100000:2.0:1789302835.055547:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5846:x1876219371154816:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302835.055610:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5846:x1876219371154816:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 66us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302835.055616:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 61 00000100:00080000:2.0:1789302835.055633:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302835.055658:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371154816:0@lo:1000:mdt_rdpg00_000.0 00000001:00080000:2.0:1789302835.056047:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968156 00000100:00100000:2.0:1789302835.056101:0:5846:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371154944:0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302835.056129:0:5846:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302835.056145:0:5846:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9bd80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302835.056156:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371154944 00000100:00100000:2.0:1789302835.056159:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371154944 00000100:00100000:2.0:1789302835.056180:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 62 00000100:00100000:2.0:1789302835.056186:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5846:x1876219371154944:12345-0@lo:1000:mdt_rdpg00_000.0 00000100:00100000:2.0:1789302835.056477:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:5846:x1876219371154944:12345-0@lo:1000:mdt_rdpg00_000.0 Request processed in 290us (346us total) trans 4294967372 rc 0/0 00000100:00100000:2.0:1789302835.056486:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 62 00000100:00100000:2.0:1789302835.056538:0:5846:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_000:lustre-MDT0000-mdtlov_UUID:5846:1876219371154944:0@lo:1000:mdt_rdpg00_000.0 00010000:00010000:2.0:1789302835.056555:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.056562:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.056570:0:5846:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.056575:0:5846:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302835.056578:0:5846:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.056587:0:5846:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.056600:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:2.0:1789302835.056603:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:2.0:1789302835.056607:0:5846:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302835.056613:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed13 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302835.056770:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x4c:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:2.0:1789302835.056893:0:5846:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].12, 64 off9032 00000004:00080000:2.0:1789302835.056905:0:5846:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 672067402205ns, next at 673067399666ns, rc = 0 00000001:00080000:2.0:1789302835.056911:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968157 00010000:00010000:2.0:1789302835.056944:0:5846:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f2c94d700) 00010000:00010000:2.0:1789302835.056948:0:5846:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:2.0:1789302835.056954:0:5846:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184ecfe lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b14 expref: 109 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302835.057031:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076eb850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30781:x1876219350451456:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 27262us (28504us total) trans 4294968158 rc 0/0 00000100:00100000:2.0:1789302835.057039:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 260 00000020:00080000:2.0:1789302835.057043:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1789302835.091040:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302835.091233:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350452096 00000100:00100000:0.0:1789302835.091244:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350452096 00000100:00100000:0.0:1789302835.091299:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1291 00000100:00100000:0.0:1789302835.091308:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:30784:x1876219350452096:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302835.091336:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302835.091357:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ed1a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b1b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091425:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091450:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091461:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091475:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091482:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091488:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091825:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ed2f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091835:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ed2f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091843:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ed2f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302835.091852:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:0.0:1789302835.091943:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091951:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091958:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091964:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302835.091969:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091976:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.091988:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:0.0:1789302835.091991:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302835.091995:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092001:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045000/0x64fe43095184ed21 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092012:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092018:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092025:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092035:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302835.092038:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092044:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092049:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f09ae1100) 00010000:00010000:0.0:1789302835.092052:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302835.092054:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092059:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed28 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092951:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ed1a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b1b expref: 111 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092969:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ed2f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b1b expref: 110 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.092978:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302835.092981:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302835.092984:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045c00, rc 0) 00000100:00100000:0.0:1789302835.093056:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:30784:x1876219350452096:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1747us (2023us total) trans 4294968159 rc 0/0 00000100:00100000:0.0:1789302835.093074:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1291 00000100:00100000:3.0:1789302835.100128:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302835.100164:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350452224 00000100:00100000:3.0:1789302835.100174:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350452224 00000100:00100000:3.0:1789302835.100222:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1292 00000100:00100000:3.0:1789302835.100230:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:30784:x1876219350452224:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:3.0:1789302835.100308:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100323:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100334:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100361:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100368:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100374:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100380:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302835.100384:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100391:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100402:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302835.100405:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302835.100409:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.100414:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed3d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302835.100476:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:30784:x1876219350452224:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 245us (357us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302835.100483:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1292 00000100:00100000:2.0:1789302835.104624:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302835.104846:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350452480 00000100:00100000:3.0:1789302835.104856:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350452480 00000100:00100000:3.0:1789302835.104986:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:3.0:1789302835.104999:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:30784:x1876219350452480:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:3.0:1789302835.105216:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:30784:x1876219350452480:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 219us (602us total) trans 4294968160 rc 0/0 00000100:00100000:3.0:1789302835.105224:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 261 00000100:00100000:3.0:1789302835.313623:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302835.316151:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350452608 00000100:00100000:1.0:1789302835.316165:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350452608 00000100:00100000:1.0:1789302835.316237:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1293 00000100:00100000:1.0:1789302835.316247:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30759:x1876219350452608:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:1.0:1789302835.316360:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30759:x1876219350452608:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 114us (2747us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302835.316367:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1293 00000100:00100000:2.0:1789302835.322608:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302835.322714:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350452864 00000100:00100000:0.0:1789302835.322725:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350452864 00000100:00100000:0.0:1789302835.322784:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 262 00000100:00100000:0.0:1789302835.322794:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30759:x1876219350452864:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302835.323048:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30759:x1876219350452864:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 254us (448us total) trans 4294968161 rc 0/0 00000100:00100000:0.0:1789302835.323055:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 262 00000100:00100000:3.0:1789302835.538310:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302835.538551:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350453120 00000100:00100000:0.0:1789302835.538561:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350453120 00000100:00100000:0.0:1789302835.538619:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1294 00000100:00100000:0.0:1789302835.538630:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:30806:x1876219350453120:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:0.0:1789302835.538658:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302835.538678:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ed44 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b22 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.538791:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed4b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.538815:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed4b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.538827:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed4b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302835.538838:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:0.0:1789302835.538990:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044400/0x64fe43095184ed44 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b22 expref: 114 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.539011:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed4b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b22 expref: 113 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302835.539020:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302835.539023:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302835.539027:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044000, rc 0) 00000100:00100000:0.0:1789302835.539289:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:30806:x1876219350453120:12345-192.168.203.11@tcp:101:cat.0 Request processed in 660us (988us total) trans 4294968162 rc 0/0 00000100:00100000:0.0:1789302835.539296:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1294 00000100:00100000:2.0:1789302835.546552:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302835.546673:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350453376 00000100:00100000:3.0:1789302835.546683:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350453376 00000100:00100000:3.0:1789302835.546747:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 57 00000100:00100000:3.0:1789302835.546758:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:30806:x1876219350453376:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:3.0:1789302835.546773:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1789302835.546796:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ed4b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b22 expref: 113 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.546829:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ed2f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b1b expref: 112 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.546841:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302835.546853:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302835.546916:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:30806:x1876219350453376:12345-192.168.203.11@tcp:103:cat.0 Request processed in 159us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302835.546923:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 57 00000100:00100000:2.0:1789302835.556228:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302835.556297:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350453504 00000100:00100000:3.0:1789302835.556306:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350453504 00000100:00100000:3.0:1789302835.556362:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1295 00000100:00100000:3.0:1789302835.556372:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:30806:x1876219350453504:12345-192.168.203.11@tcp:58:cat.0 00010000:00010000:3.0:1789302835.556472:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.556489:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.556548:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302835.556710:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].38, 136 off13360 00000100:00100000:3.0:1789302835.556824:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:30806:x1876219350453504:12345-192.168.203.11@tcp:58:cat.0 Request processed in 452us (605us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302835.556832:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1295 00000100:00100000:2.0:1789302835.560610:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302835.560738:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350453632 00000100:00100000:3.0:1789302835.560748:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350453632 00000100:00100000:3.0:1789302835.560863:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1296 00000100:00100000:3.0:1789302835.560874:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:30806:x1876219350453632:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302835.560896:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302835.560917:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b29 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.560973:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.560985:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b29 expref: 112 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.561004:0:6919:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.561021:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b29 expref: 112 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.561034:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302835.561038:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302835.561041:0:6919:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b29 expref: 112 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302835.561048:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1000, rc 0) 00000100:00100000:3.0:1789302835.561111:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:30806:x1876219350453632:12345-192.168.203.11@tcp:101:cat.0 Request processed in 239us (510us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302835.561118:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1296 00000040:00080000:2.0:1789302835.853140:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302835.853148:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302835.853190:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 6 00000040:00080000:2.0:1789302835.853197:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302835.853836:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:2.0:1789302836.811056:0:6719:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00010000:00010000:1.0:1789302836.876820:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302837.260810:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371155328:0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.260877:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302837.260944:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1789302837.260951:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371155200:0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.260974:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302837.260993:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371155712:0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.261015:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302837.261051:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155328 00000100:00100000:2.0:1789302837.261058:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155328 00000100:00100000:2.0:1789302837.261109:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155200 00000100:00100000:2.0:1789302837.261112:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155200 00000100:00100000:2.0:1789302837.261133:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:2.0:1789302837.261141:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371155328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.261189:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371155328:12345-0@lo:400:kworker.0 Request processed in 48us (312us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302837.261196:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:2.0:1789302837.261212:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:2.0:1789302837.261217:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371155200:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.261248:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371155200:12345-0@lo:400:kworker.0 Request processed in 31us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302837.261254:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:2.0:1789302837.261275:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155712 00000100:00100000:2.0:1789302837.261278:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155712 00000100:00100000:2.0:1789302837.261310:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1789302837.261317:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371155712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.261347:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371155712:12345-0@lo:400:kworker.0 Request processed in 30us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302837.261353:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00080000:2.0:1789302837.261369:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302837.261373:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302837.261406:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371155200:0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.261417:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371155712:0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.261425:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe32300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371155328:0@lo:400:kworker.0 00000100:00100000:3.0:1789302837.261594:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371155456:0@lo:400:kworker.0 00000100:00100000:3.0:1789302837.261625:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302837.261654:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1789302837.261660:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371155072:0@lo:400:kworker.0 00000100:00100000:3.0:1789302837.261686:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302837.261709:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371155584:0@lo:400:kworker.0 00000100:00100000:3.0:1789302837.261731:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302837.261755:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155072 00000100:00100000:3.0:1789302837.261759:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155072 00000100:00100000:3.0:1789302837.261784:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 601 00000100:00100000:3.0:1789302837.261791:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371155072:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302837.261835:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371155072:12345-0@lo:400:kworker.0 Request processed in 44us (148us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302837.261842:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 601 00000100:00100000:3.0:1789302837.261866:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155584 00000100:00100000:3.0:1789302837.261870:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155584 00000100:00100000:3.0:1789302837.261893:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1789302837.261899:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371155584:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302837.261933:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371155584:12345-0@lo:400:kworker.0 Request processed in 34us (201us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302837.261938:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:3.0:1789302837.261962:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371155072:0@lo:400:kworker.0 00000100:00100000:3.0:1789302837.261971:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371155584:0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.262343:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155456 00000100:00100000:2.0:1789302837.262348:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155456 00000100:00100000:2.0:1789302837.262384:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1789302837.262392:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371155456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302837.262439:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371155456:12345-0@lo:400:kworker.0 Request processed in 48us (814us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302837.262455:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00080000:2.0:1789302837.262471:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302837.262474:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302837.263286:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe33100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371155456:0@lo:400:kworker.0 00000004:00080000:3.0:1789302837.772721:0:8001:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 674783216295ns, next at 675783214045ns, rc = 0 00000100:00100000:1.0:1789302837.772979:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302837.773000:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371155840:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302837.773075:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302837.773279:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155840 00000100:00100000:3.0:1789302837.773287:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155840 00000100:00100000:3.0:1789302837.773339:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:3.0:1789302837.773356:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371155840:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302837.773387:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3855613952 avail 3853516800 00000100:00100000:3.0:1789302837.773436:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371155840:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (360us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302837.773443:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:1.0:1789302837.773619:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371155840:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302837.901038:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302838.286109:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302838.286132:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371155968:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302838.286200:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302838.287387:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371155968 00000100:00100000:2.0:1789302838.287399:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371155968 00000100:00100000:2.0:1789302838.287467:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 199 00000100:00100000:2.0:1789302838.287478:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371155968:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789302838.287549:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789302838.287757:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371155968:12345-0@lo:13:osp-pre-0-0.0 Request processed in 281us (1556us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302838.287765:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 199 00000100:00100000:3.0:1789302838.287858:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371155968:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302838.541875:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.541964:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.541996:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.542027:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302838.542092:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454144 00000100:00100000:0.0:1789302838.542103:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454144 00000100:00100000:0.0:1789302838.542167:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454400 00000100:00100000:0.0:1789302838.542170:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454400 00000100:00100000:0.0:1789302838.542193:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 472 00000100:00100000:0.0:1789302838.542203:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350454144:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302838.542274:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350454144:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 73us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302838.542282:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 472 00000100:00100000:0.0:1789302838.542300:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00100000:0.0:1789302838.542307:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350454400:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302838.542330:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350454400:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302838.542336:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00080000:0.0:1789302838.542441:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302838.542445:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302838.550218:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350453888 00000100:00100000:3.0:1789302838.550233:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350453888 00000100:00080000:0.0:1789302838.550391:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302838.553446:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454656 00000100:00100000:3.0:1789302838.553462:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454656 00000100:00100000:3.0:1789302838.553543:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:3.0:1789302838.553554:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2348:x1876219350453888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302838.553640:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2348:x1876219350453888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 88us (11775us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302838.553648:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:3.0:1789302838.553667:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00100000:3.0:1789302838.553673:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:2348:x1876219350454656:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302838.553704:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:2348:x1876219350454656:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (11676us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302838.553711:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00080000:0.0:1789302838.555817:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302838.556904:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.556947:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.556983:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.557023:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.557062:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454272 00000100:00100000:2.0:1789302838.557071:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454272 00000100:00100000:2.0:1789302838.557126:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 465 00000100:00100000:2.0:1789302838.557136:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350454272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302838.557202:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350454272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 67us (178us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302838.557209:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 465 00000100:00100000:2.0:1789302838.557244:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302838.557347:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302838.557557:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454912 00000100:00100000:0.0:1789302838.557562:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454912 00000100:00100000:0.0:1789302838.557605:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454016 00000100:00100000:0.0:1789302838.557609:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454016 00000100:00100000:0.0:1789302838.557627:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454528 00000100:00100000:0.0:1789302838.557630:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454528 00000100:00100000:0.0:1789302838.557640:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350454784 00000100:00100000:0.0:1789302838.557642:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350454784 00000100:00100000:0.0:1789302838.557657:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 474 00000100:00100000:0.0:1789302838.557664:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350454912:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302838.557702:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350454912:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (805us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302838.557708:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 474 00000100:00100000:0.0:1789302838.557722:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 475 00000100:00100000:0.0:1789302838.557728:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350454016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302838.557753:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:2348:x1876219350454016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (769us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302838.557759:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 475 00000100:00100000:0.0:1789302838.557779:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 476 00000100:00100000:0.0:1789302838.557797:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350454528:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302838.557822:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350454528:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302838.557827:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 476 00000100:00100000:0.0:1789302838.557838:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 477 00000100:00100000:0.0:1789302838.557845:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350454784:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302838.557864:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350454784:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (518us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302838.557870:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 477 00000100:00080000:0.0:1789302838.557950:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302838.557954:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302838.557956:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302838.561120:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350453760 00000100:00100000:3.0:1789302838.561136:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350453760 00000100:00100000:3.0:1789302838.561203:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 602 00000100:00100000:3.0:1789302838.561213:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:2348:x1876219350453760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302838.562069:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:2348:x1876219350453760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 855us (5118us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302838.562076:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 602 00000001:00080000:0.0:1789302838.867741:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968162 is committed 00000001:00080000:0.0:1789302838.867761:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968161 is committed 00000004:00080000:0.0:1789302838.868122:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 13 committed 00000020:00080000:0.0:1789302838.868142:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f3ff9b240 refcount 2 committed 0 result 0 batchid 29 00000020:00080000:0.0:1789302838.868147:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f085afc00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f33ff9180 00000020:00080000:0.0:1789302838.868150:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f1bd4c400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3f048800 00000020:00080000:3.0:1789302838.868188:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 29 committed batchid 28 00000020:00080000:3.0:1789302838.868385:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 28 00000100:00100000:0.0:1789302838.869089:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000004:00080000:1.0:1789302838.869256:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 14 committed 00000001:00080000:1.0:1789302838.869274:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968146 is committed 00000100:00100000:0.0:1789302838.869353:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371156096:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789302838.870142:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302838.870306:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371156096 00000100:00100000:3.0:1789302838.870315:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371156096 00000100:00100000:3.0:1789302838.870851:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:3.0:1789302838.870874:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371156096:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1789302838.870906:0:6695:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:26 count 1 00002000:00010000:3.0:1789302838.870941:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1a:0x0] initial LVB size: 1048576, mtime: 0x6aa69831, atime: 0x0, ctime: 0x6aa69831, blocks: 0x803 00010000:00010000:3.0:1789302838.870950:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302838.870966:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=675s v=(0 0) 00010000:00010000:3.0:1789302838.870970:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1a:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302838.870976:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302838.870992:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302838.871001:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302838.871008:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302838.871036:0:6695:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302838.872107:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371156096:12345-0@lo:6:osp-syn-1-0.0 Request processed in 1234us (1964us total) trans 4294967373 rc 0/0 00000100:00100000:3.0:1789302838.872117:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00010000:00010000:1.0:1789302838.872245:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302838.872253:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302838.872262:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302838.872266:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302838.872275:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302838.872292:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302838.872299:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184ed67 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302838.872321:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302838.872323:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302838.872324:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789302838.877771:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f63d500/2, rc 0, transno 4294967373 00000100:00100000:2.0:1789302838.877794:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371156096:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1789302838.924988:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302839.385456:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967372 is committed 00000001:00080000:2.0:1789302839.385474:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967373 is committed 00010000:00010000:3.0:1789302839.948832:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302840.561057:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302840.561293:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350455040 00000100:00100000:3.0:1789302840.561303:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350455040 00000100:00100000:3.0:1789302840.561434:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1297 00000100:00100000:3.0:1789302840.561446:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30808:x1876219350455040:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:3.0:1789302840.561470:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302840.561491:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ed6e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b30 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561597:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561612:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561626:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561640:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561647:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561654:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561966:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561976:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x53/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.561983:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x53/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302840.562020:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:3.0:1789302840.562024:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302840.562045:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x53/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562055:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562062:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562067:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562073:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562079:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302840.562083:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562090:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562101:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:3.0:1789302840.562103:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302840.562108:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562114:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ed75 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562126:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562137:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562142:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562147:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302840.562149:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562154:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562161:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302840.562163:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302840.562166:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562171:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ed7c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562202:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ed6e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562215:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 47 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302840.562222:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302840.562224:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302840.562227:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302840.562296:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30808:x1876219350455040:12345-192.168.203.11@tcp:101:stat.0 Request processed in 853us (1248us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302840.562304:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1297 00000100:00080000:0.0:1789302840.562373:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302840.972616:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302840.972691:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302840.972697:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302840.973223:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 3 00000040:00080000:2.0:1789302840.973231:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302841.032624:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302841.032733:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350455168 00000100:00100000:2.0:1789302841.032751:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350455168 00000100:00100000:2.0:1789302841.032854:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1298 00000100:00100000:2.0:1789302841.032869:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30832:x1876219350455168:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302841.032887:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302841.032908:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ed8a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b37 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302841.032965:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ed91 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302841.032985:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ed91 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302841.032998:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ed91 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302841.033030:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302841.033033:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302841.033050:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ed91 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302841.033071:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184ed8a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b37 expref: 9 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302841.033090:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ed91 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b37 expref: 8 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302841.033099:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:2.0:1789302841.033102:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302841.033105:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302841.033166:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:30832:x1876219350455168:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 304us (556us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302841.033173:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1298 00000100:00080000:0.0:1789302841.033866:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302841.042443:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302841.042736:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350455424 00000100:00100000:1.0:1789302841.042747:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350455424 00000100:00100000:1.0:1789302841.042822:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1299 00000100:00100000:1.0:1789302841.042833:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:30833:x1876219350455424:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302841.042855:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302841.042875:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ed98 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b3e expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.042943:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 3/1,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.042958:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.042969:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.042985:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.042992:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.042999:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043255:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184edad lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043273:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184edad lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043282:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184edad lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302841.043292:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000002:0x2:0x0]: returning size 16384 00010000:00010000:1.0:1789302841.043419:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043427:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043435:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043441:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302841.043445:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043454:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043466:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:1.0:1789302841.043469:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302841.043475:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 2/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043487:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184ed9f lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043569:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043577:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043582:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043586:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302841.043589:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043595:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043603:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x8d45d6bc (ffff960f2c94d900) 00010000:00010000:1.0:1789302841.043606:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302841.043608:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043615:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184eda6 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x8d45d6bc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.043994:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ed98 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b3e expref: 11 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.044012:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184edad lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b3e expref: 10 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302841.044021:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:1.0:1789302841.044024:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302841.044027:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc 0) 00000100:00100000:1.0:1789302841.044092:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30833:x1876219350455424:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1260us (1659us total) trans 4294968163 rc 0/0 00000100:00100000:1.0:1789302841.044101:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1299 00000100:00100000:3.0:1789302841.050955:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302841.051014:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302841.051049:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350455552 00000100:00100000:2.0:1789302841.051065:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350455552 00000100:00100000:2.0:1789302841.051143:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1300 00000100:00100000:2.0:1789302841.051154:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30833:x1876219350455552:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302841.051293:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30833:x1876219350455552:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 138us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302841.051301:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1300 00000100:00100000:0.0:1789302841.051448:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350455680 00000100:00100000:0.0:1789302841.051459:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350455680 00000100:00100000:3.0:1789302841.051607:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 58 00000100:00100000:3.0:1789302841.051619:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2345:x1876219350455680:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302841.051631:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302841.051722:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184edad lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b3e expref: 10 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302841.051741:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302841.051746:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302841.051795:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:2345:x1876219350455680:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 180us (781us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302841.051801:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 58 00010000:00010000:1.0:1789302841.997424:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302841.998043:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302841.998052:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302841.998233:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 3 00000040:00080000:2.0:1789302841.998242:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302841.998425:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c46680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371156224:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302841.998698:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9bd80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302842.003401:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.007307:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350455808 00000100:00100000:0.0:1789302842.007330:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350455808 00000100:00100000:0.0:1789302842.007425:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1301 00000100:00100000:0.0:1789302842.007438:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30834:x1876219350455808:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302842.007783:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302842.008301:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c46680 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371156224:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302842.008819:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30834:x1876219350455808:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 1379us (5430us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.008842:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1301 00000100:00100000:1.0:1789302842.017983:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.025113:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350455936 00000100:00100000:2.0:1789302842.025134:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350455936 00000100:00100000:2.0:1789302842.025314:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1302 00000100:00100000:2.0:1789302842.025329:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30834:x1876219350455936:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:2.0:1789302842.025483:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025544:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025562:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025586:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025595:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025608:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025651:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025662:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025669:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302842.025731:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:2.0:1789302842.025738:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302842.025775:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025789:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025796:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025804:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025811:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.025816:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025825:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025840:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302842.025844:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.025850:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025858:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edbb lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025873:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025880:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025886:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025892:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.025895:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025901:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025909:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x4e3552f8 (ffff960f10d3c500) 00010000:00010000:2.0:1789302842.025911:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.025915:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025921:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edc2 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x4e3552f8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025944:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025951:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025958:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025963:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.025966:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025973:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025981:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:2.0:1789302842.025984:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.025987:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.025994:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edc9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.026120:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30834:x1876219350455936:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 793us (8665us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.026130:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1302 00000100:00100000:3.0:1789302842.033249:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.033795:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350456192 00000100:00100000:2.0:1789302842.033809:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350456192 00000100:00100000:2.0:1789302842.033923:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1303 00000100:00100000:2.0:1789302842.033936:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30834:x1876219350456192:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302842.033963:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302842.033990:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edd0 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b45 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034129:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034148:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034162:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302842.034175:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:2.0:1789302842.034305:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034314:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034321:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034327:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.034332:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034341:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034354:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:2.0:1789302842.034358:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302842.034364:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034371:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184edd7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034437:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edd0 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b45 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034449:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302842.034451:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302842.034455:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184edd0 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b45 expref: 11 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.034472:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:2.0:1789302842.034575:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30834:x1876219350456192:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 641us (2145us total) trans 4294968164 rc 301/301 00000100:00100000:2.0:1789302842.034584:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1303 00000100:00100000:1.0:1789302842.047063:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.047357:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350456320 00000100:00100000:0.0:1789302842.047372:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350456320 00000100:00100000:3.0:1789302842.048045:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1304 00000100:00100000:3.0:1789302842.048064:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30834:x1876219350456320:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302842.048115:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x27:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302842.048904:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30834:x1876219350456320:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 842us (1851us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.048914:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1304 00000100:00100000:1.0:1789302842.058025:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.059815:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350456448 00000100:00100000:2.0:1789302842.059832:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350456448 00000100:00100000:2.0:1789302842.059930:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1305 00000100:00100000:2.0:1789302842.059942:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30834:x1876219350456448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302842.059971:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302842.060000:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ede5 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b4c expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060085:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060103:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060116:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060133:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060141:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060148:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060809:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184edfa lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060823:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184edfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060832:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184edfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302842.060871:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4b:0x0]: returning size 16384 00000004:00200000:2.0:1789302842.060875:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302842.060900:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184edfa lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060915:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060928:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060935:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060941:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.060946:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060953:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060966:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:2.0:1789302842.060969:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.060974:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060981:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184edec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.060994:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061000:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061006:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061011:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.061014:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061020:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061026:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x28683846 (ffff960f10d3c500) 00010000:00010000:2.0:1789302842.061029:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.061032:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061038:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184edf3 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x28683846 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061078:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ede5 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b4c expref: 12 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061092:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184edfa lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b4c expref: 11 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.061100:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:2.0:1789302842.061103:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302842.061105:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2a00, rc 0) 00000100:00100000:2.0:1789302842.061201:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350456448:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1261us (3193us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.061210:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1305 00000100:00100000:2.0:1789302842.065002:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.065233:0:5853:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350456576 00000100:00100000:0.0:1789302842.065247:0:5853:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350456576 00000100:00100000:0.0:1789302842.065321:0:5853:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 2 00000100:00100000:0.0:1789302842.065334:0:5853:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqm_0001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350456576:12345-192.168.203.11@tcp:700:lhsmtool_posix.0 00000100:00100000:0.0:1789302842.065897:0:5853:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_seqm_0001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350456576:12345-192.168.203.11@tcp:700:lhsmtool_posix.0 Request processed in 563us (907us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.065908:0:5853:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 2 00000100:00100000:1.0:1789302842.070252:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.070413:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350456832 00000100:00100000:2.0:1789302842.070593:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350456832 00000100:00100000:2.0:1789302842.070710:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1306 00000100:00100000:2.0:1789302842.070723:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350456832:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302842.070772:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302842.070801:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee01 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b53 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.072680:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee08 lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.072703:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee08 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.073008:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee08 lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302842.073026:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1:0x0]: returning size 0 00010000:00010000:2.0:1789302842.073107:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee01 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b53 expref: 14 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.073131:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee08 lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.073142:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:2.0:1789302842.073146:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302842.073150:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302842.073333:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:30834:x1876219350456832:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2611us (3096us total) trans 4294968165 rc 0/0 00000100:00100000:2.0:1789302842.073344:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1306 00000100:00100000:1.0:1789302842.078452:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.078812:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350457088 00000100:00100000:3.0:1789302842.078825:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350457088 00000100:00100000:3.0:1789302842.079056:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1307 00000100:00100000:3.0:1789302842.079070:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:30834:x1876219350457088:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302842.079153:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302842.079188:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee08 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b53 expref: 13 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.079211:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:3.0:1789302842.079224:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302842.079255:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.079272:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.079282:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.079327:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.079343:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 99 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.079362:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371156352:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302842.079566:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe00c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302842.083091:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371156352:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302842.083104:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302842.083132:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.083197:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350457216 00000100:00100000:0.0:1789302842.083207:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350457216 00010000:00010000:0.0:1789302842.083317:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 100 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.083364:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 388 00000100:00100000:0.0:1789302842.083376:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f09750850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:2348:x1876219350457216:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302842.083396:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.083405:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 100 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.083430:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 100 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.083459:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095184eca3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0aea expref: 100 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.083482:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:0.0:1789302842.083488:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.083526:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.083536:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.083566:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302842.083620:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.083629:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.083635:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f09750850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:2348:x1876219350457216:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 262us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.083642:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 388 00010000:00010000:3.0:1789302842.083883:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.083905:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.083914:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084247:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084268:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084276:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084283:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302842.084289:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084298:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084313:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:3.0:1789302842.084316:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302842.084321:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084328:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184ee16 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084341:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084347:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084352:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084358:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302842.084361:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084367:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084375:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa4e7bbdc (ffff960f0cdf2700) 00010000:00010000:3.0:1789302842.084377:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302842.084380:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.084385:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee1d lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4b:0x0].0xa4e7bbdc bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.084746:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+12:30834:x1876219350457088:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 5676us (6320us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302842.084755:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1307 00000100:00100000:2.0:1789302842.090379:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00080000:1.0:1789302842.091301:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302842.091653:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350457344 00000100:00100000:3.0:1789302842.091668:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350457344 00000100:00100000:3.0:1789302842.091746:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1308 00000100:00100000:3.0:1789302842.091758:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+12:30834:x1876219350457344:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302842.091797:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302842.091828:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ee24 lrc: 2/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b5a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.091887:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.091906:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.092069:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.092195:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ee24 lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 14 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.092217:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 13 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.092226:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302842.092230:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302842.092234:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302842.092327:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:30834:x1876219350457344:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 571us (1961us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.092335:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1308 00000100:00100000:0.0:1789302842.096336:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.096452:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350457600 00000100:00100000:2.0:1789302842.096465:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350457600 00000100:00100000:2.0:1789302842.096578:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1309 00000100:00100000:2.0:1789302842.096591:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:30834:x1876219350457600:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302842.096712:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.096734:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.096749:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302842.097055:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002f5d2300 reply state 000000001b487979 transno 4294968166 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.097066:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302842.097183:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+15:30834:x1876219350457600:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 594us (860us total) trans 4294968166 rc 0/0 00000100:00100000:2.0:1789302842.097191:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1309 00000100:00100000:0.0:1789302842.102447:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302842.102550:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.102566:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.102574:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.102582:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302842.102587:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.102595:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.102617:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302842.102620:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:3.0:1789302842.102627:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 2/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.102633:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee32 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.102826:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350457728 00000100:00100000:2.0:1789302842.102850:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350457728 00000100:00100000:0.0:1789302842.103012:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1310 00000100:00100000:0.0:1789302842.103026:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+14:30834:x1876219350457728:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302842.103058:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.103083:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b61 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.103157:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.103174:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 15 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.103189:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302842.103191:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302842.103195:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42045c00, rc 0) 00000100:00100000:0.0:1789302842.103291:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+15:30834:x1876219350457728:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 268us (867us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.103299:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1310 00000100:00100000:3.0:1789302842.106944:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.107120:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350457856 00000100:00100000:0.0:1789302842.107135:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350457856 00000100:00100000:0.0:1789302842.107209:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:0.0:1789302842.107221:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+15:30834:x1876219350457856:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:0.0:1789302842.107401:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+15:30834:x1876219350457856:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 181us (472us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.107409:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 263 00000100:00100000:1.0:1789302842.110868:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.111012:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350458112 00000100:00100000:3.0:1789302842.111028:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350458112 00000100:00100000:3.0:1789302842.111102:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 264 00000100:00100000:3.0:1789302842.111113:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+15:30834:x1876219350458112:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302842.111434:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350458112:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 321us (581us total) trans 4294968167 rc 0/0 00000100:00100000:3.0:1789302842.111442:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 264 00000100:00100000:1.0:1789302842.118329:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.118609:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350458368 00000100:00100000:3.0:1789302842.118624:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350458368 00000100:00100000:3.0:1789302842.118705:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1311 00000100:00100000:3.0:1789302842.118718:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350458368:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302842.118893:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350458368:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 177us (577us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.118901:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1311 00000100:00100000:2.0:1789302842.122075:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302842.122274:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350458624 00000100:00100000:1.0:1789302842.122290:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350458624 00000100:00100000:1.0:1789302842.122375:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1312 00000100:00100000:1.0:1789302842.122385:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350458624:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302842.122426:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302842.122458:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184ee40 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b68 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.122656:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.122676:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 2/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.122687:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.122734:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 17 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.122753:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 17 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302842.122780:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371156480:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302842.122889:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6480 going to sleep for 11 seconds 00000100:00100000:1.0:1789302842.129361:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.129491:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350458752 00000100:00100000:0.0:1789302842.129541:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350458752 00010000:00010000:0.0:1789302842.129594:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 18 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.129641:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 389 00000100:00100000:0.0:1789302842.129652:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:2348:x1876219350458752:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302842.129671:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.129688:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 18 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.129705:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 18 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.129733:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302842.129739:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.129752:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.129768:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.129796:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302842.129865:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:2348:x1876219350458752:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 214us (518us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.129872:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 389 00010000:00010000:3.0:1789302842.131242:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee39 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b61 expref: 17 pid: 5845 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.131282:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90000 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371156480:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302842.131314:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.131325:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.131331:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302842.131359:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1:0x0]: returning size 0 00000004:00080000:3.0:1789302842.132084:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1b:0x0], index 0: 0 00010000:00010000:3.0:1789302842.132293:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132303:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132310:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132316:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302842.132320:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132328:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132347:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302842.132351:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302842.132356:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132363:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184ee47 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132434:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184ee40 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b68 expref: 17 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132446:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:3.0:1789302842.132449:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302842.132453:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184ee40 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b68 expref: 17 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.132468:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8c00, rc 0) 00000100:00100000:3.0:1789302842.132590:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350458624:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 10204us (10527us total) trans 4294968168 rc 301/301 00000100:00100000:3.0:1789302842.132601:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1312 00000100:00100000:1.0:1789302842.136776:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.136999:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350459008 00000100:00100000:3.0:1789302842.137013:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350459008 00000100:00100000:3.0:1789302842.137116:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 265 00000100:00100000:3.0:1789302842.137128:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350459008:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302842.137657:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+17:30834:x1876219350459008:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 529us (902us total) trans 4294968169 rc 0/0 00000100:00100000:3.0:1789302842.137666:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 265 00000100:00100000:2.0:1789302842.145630:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.145747:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350459136 00000100:00100000:3.0:1789302842.145767:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350459136 00000100:00100000:2.0:1789302842.145935:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1313 00000100:00100000:2.0:1789302842.145954:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+17:30834:x1876219350459136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302842.145987:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302842.146014:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ee55 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b6f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.146248:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee5c lrc: 3/1,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.146279:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee5c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.146306:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee5c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302842.146395:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000403:0x1:0x0]: returning size 0 00000004:00200000:2.0:1789302842.146406:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302842.146567:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee5c lrc: 3/1,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.146599:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ee55 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b6f expref: 19 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.146687:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee5c lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b6f expref: 18 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.146702:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:2.0:1789302842.146708:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:2.0:1789302842.146716:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302842.147003:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:30834:x1876219350459136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1054us (1385us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.147016:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1313 00000100:00100000:2.0:1789302842.150705:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.150815:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350459264 00000100:00100000:0.0:1789302842.150886:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350459264 00000100:00100000:0.0:1789302842.151071:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 361 00000100:00100000:0.0:1789302842.151085:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f08598050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2347:x1876219350459264:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302842.151108:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.151135:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b76 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302842.151168:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302842.151196:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=679s v=(0 0) 00010000:00010000:0.0:1789302842.151201:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302842.151212:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 6 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.151239:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 6 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.151254:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f42044400, rc 0) 00000100:00100000:0.0:1789302842.151337:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f08598050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2347:x1876219350459264:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 257us (647us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.151345:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 361 00000100:00080000:1.0:1789302842.151947:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302842.158494:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.158707:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350459520 00000100:00100000:2.0:1789302842.158722:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350459520 00000100:00100000:2.0:1789302842.158848:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1314 00000100:00100000:2.0:1789302842.158860:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:30834:x1876219350459520:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302842.159038:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:30834:x1876219350459520:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 179us (625us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.159045:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1314 00000100:00100000:0.0:1789302842.167863:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302842.168175:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350459648 00000100:00100000:1.0:1789302842.168185:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350459648 00000100:00100000:1.0:1789302842.168278:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:1.0:1789302842.168288:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:30834:x1876219350459648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302842.168305:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302842.168334:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b7d expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.168366:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 7 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.168381:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302842.168385:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302842.168390:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.168423:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 7 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.168438:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 7 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302842.168453:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371156608:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302842.168941:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302842.178769:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371156608:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302842.178786:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:0.0:1789302842.178813:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->1048575] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.178841:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8200, rc 0) 00000100:00100000:0.0:1789302842.178921:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:30834:x1876219350459648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 10634us (11069us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.178930:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 362 00000100:00100000:3.0:1789302842.184095:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350459776 00000100:00100000:3.0:1789302842.184118:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350459776 00010000:00010000:3.0:1789302842.184204:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 7 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.184301:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:3.0:1789302842.184312:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:2347:x1876219350459776:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302842.184332:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302842.184359:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 7 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.184382:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 7 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.184412:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000628dfe70/0x64fe43095184ee63 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b76 expref: 7 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.184445:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302842.184450:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302842.184460:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.184470:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.184558:0:8463:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 6 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.184575:0:8463:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8463:1876219371156736:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302842.184692:0:8463:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302842.198864:0:8463:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_003:lustre-OST0000_UUID:8463:1876219371156736:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302842.198903:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302842.199856:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2347:x1876219350459776:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 15537us (21070us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.199872:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:1.0:1789302842.219796:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.219969:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350459904 00000100:00100000:3.0:1789302842.219983:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350459904 00002000:00100000:3.0:1789302842.220055:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x1b:0x0] (0->1048575) req@ffff960f41c90e00 x1876219350459904/t0(0) o4->36d456e5-2a1b-4d58-a7f1-78ab3c35acb0@192.168.203.11@tcp:418/0 lens 488/0 e 0 to 0 dl 1789302853 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302842.220071:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c90e00 with x1876219350459904 ext(0->1048575) 00002000:00010000:3.0:1789302842.220077:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41c90e00 00000100:00100000:3.0:1789302842.220159:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 115 00000100:00100000:3.0:1789302842.220171:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2346:x1876219350459904:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302842.258365:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:2346:x1876219350459904:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 38174us (38560us total) trans 4294967370 rc 0/0 00000100:00100000:2.0:1789302842.258387:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 115 00002000:00100000:2.0:1789302842.258408:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x1b:0x0] (0->1048575) req@ffff960f41c90e00 x1876219350459904/t4294967370(0) o4->36d456e5-2a1b-4d58-a7f1-78ab3c35acb0@192.168.203.11@tcp:418/0 lens 488/448 e 0 to 0 dl 1789302853 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302842.258434:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c90e00 with x1876219350459904 ext(0->1048575) 00002000:00010000:2.0:1789302842.258443:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41c90e00 00000100:00100000:2.0:1789302842.267206:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.268299:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350460032 00000100:00100000:3.0:1789302842.268319:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350460032 00000100:00100000:3.0:1789302842.268413:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1315 00000100:00100000:3.0:1789302842.268425:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:30834:x1876219350460032:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1789302842.368813:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968163 is committed 00000001:00080000:3.0:1789302842.368834:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968166 is committed 00000001:00080000:3.0:1789302842.368851:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968169 is committed 00000020:00080000:3.0:1789302842.368861:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 29 batchid updated 00000020:00080000:3.0:1789302842.368877:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 29 00000020:00080000:3.0:1789302842.368893:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 29 00000100:00100000:0.0:1789302842.368924:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350460032:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 100495us (101720us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.368935:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1315 00000100:00100000:0.0:1789302842.375954:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.376070:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350460160 00000100:00100000:2.0:1789302842.376212:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350460160 00000100:00100000:2.0:1789302842.376268:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 363 00000100:00100000:2.0:1789302842.376278:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:2347:x1876219350460160:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000100:00100000:2.0:1789302842.380892:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371157120:0@lo:400:kworker.0 00000100:00100000:2.0:1789302842.380959:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302842.381035:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302842.381041:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371157248:0@lo:400:kworker.0 00000100:00100000:2.0:1789302842.381068:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302842.381328:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371156992:0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381357:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302842.381377:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371157504:0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381397:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302842.381413:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789302842.381417:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371156864:0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381440:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302842.381460:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371157376:0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381477:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302842.381542:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371157120 00000100:00100000:3.0:1789302842.381560:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371157120 00000100:00100000:3.0:1789302842.381609:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371156992 00000100:00100000:3.0:1789302842.381613:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371156992 00000100:00100000:3.0:1789302842.381631:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371157376 00000100:00100000:3.0:1789302842.381633:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371157376 00000100:00100000:3.0:1789302842.381651:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789302842.381658:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371157120:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381706:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371157120:12345-0@lo:400:kworker.0 Request processed in 48us (746us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.381713:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1789302842.381730:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789302842.381735:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371156992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381778:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371156992:12345-0@lo:400:kworker.0 Request processed in 43us (420us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.381784:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1789302842.381795:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:3.0:1789302842.381801:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371157376:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381827:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371157376:12345-0@lo:400:kworker.0 Request processed in 27us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.381841:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:3.0:1789302842.381859:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371157248 00000100:00100000:3.0:1789302842.381863:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371157248 00000100:00100000:3.0:1789302842.381886:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371157504 00000100:00100000:3.0:1789302842.381889:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371157504 00000100:00100000:3.0:1789302842.381909:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:3.0:1789302842.381915:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371157248:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381944:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371157248:12345-0@lo:400:kworker.0 Request processed in 30us (876us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.381954:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:3.0:1789302842.381965:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789302842.381971:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371157504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.381997:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371157504:12345-0@lo:400:kworker.0 Request processed in 26us (599us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.382002:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:3.0:1789302842.382247:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371156864 00000100:00100000:3.0:1789302842.382251:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371156864 00000100:00100000:3.0:1789302842.382277:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 603 00000100:00100000:3.0:1789302842.382282:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371156864:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.382310:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371156864:12345-0@lo:400:kworker.0 Request processed in 28us (869us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.382316:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 603 00000100:00080000:3.0:1789302842.382341:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302842.382345:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302842.382348:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302842.382384:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371156864:0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.382397:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371157376:0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.382406:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371156992:0@lo:400:kworker.0 00000100:00100000:3.0:1789302842.382413:0:3301:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967373 gen 2 00000004:00080000:3.0:1789302842.382418:0:3301:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967372 committed transno 4294967373 00000004:00080000:3.0:1789302842.382437:0:3301:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f63d500, transno 4294967373 00000100:00100000:3.0:1789302842.382447:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371157504:0@lo:400:kworker.0 00000100:00100000:2.0:1789302842.382942:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371157248:0@lo:400:kworker.0 00000100:00100000:2.0:1789302842.382954:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371157120:0@lo:400:kworker.0 00000001:00080000:0.0:1789302842.464257:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967370 is committed 00000100:00100000:2.0:1789302842.464656:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2347:x1876219350460160:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 88363us (88694us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.464671:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 363 00000100:00100000:3.0:1789302842.468272:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.468548:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350460416 00000100:00100000:0.0:1789302842.468562:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350460416 00000100:00100000:0.0:1789302842.468639:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1316 00000100:00100000:0.0:1789302842.468650:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350460416:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302842.468718:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.468756:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ee5c lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b6f expref: 11 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.468790:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302842.468796:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302842.468845:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.468862:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.468874:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302842.469108:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000df729bd2 reply state 000000001d7cdc7d transno 4294968170 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.469120:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302842.469256:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+12:30834:x1876219350460416:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 607us (1007us total) trans 4294968170 rc 0/0 00000100:00100000:0.0:1789302842.469264:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1316 00000100:00100000:1.0:1789302842.471282:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.471609:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350460544 00000100:00100000:3.0:1789302842.471623:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350460544 00000100:00100000:3.0:1789302842.471687:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1317 00000100:00100000:3.0:1789302842.471699:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+12:30834:x1876219350460544:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302842.471728:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302842.471754:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 2/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b84 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.471851:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 13 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.471868:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 13 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.471894:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302842.471898:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302842.471903:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1400, rc 0) 00000100:00100000:3.0:1789302842.471981:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:30834:x1876219350460544:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 284us (708us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.471989:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1317 00010000:00010000:2.0:1789302842.472989:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.473006:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.473014:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.473021:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42045c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.473027:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.473036:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.473057:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:2.0:1789302842.473061:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.473066:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 2/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.473073:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045c00/0x64fe43095184ee71 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.474739:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.477190:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350460672 00000100:00100000:3.0:1789302842.477208:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350460672 00000100:00100000:3.0:1789302842.477322:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 364 00000100:00100000:3.0:1789302842.477335:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2347:x1876219350460672:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1789302842.477718:0:6695:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 0 -> 1048576 00002000:00010000:3.0:1789302842.477734:0:6695:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 0 -> 1789302833 00002000:00010000:3.0:1789302842.477737:0:6695:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb atime from disk: 0 -> 1789302833 00002000:00010000:3.0:1789302842.477740:0:6695:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 0 -> 1789302833 00002000:00010000:3.0:1789302842.477743:0:6695:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 1 -> 2051 00000100:00100000:3.0:1789302842.477851:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:2347:x1876219350460672:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 518us (3121us total) trans 4294967371 rc 0/0 00000100:00100000:3.0:1789302842.477861:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 364 00000100:00100000:1.0:1789302842.480390:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.480692:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350460928 00000100:00100000:2.0:1789302842.480708:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350460928 00000100:00100000:2.0:1789302842.480826:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1318 00000100:00100000:2.0:1789302842.480839:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+12:30834:x1876219350460928:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:2.0:1789302842.480930:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.480951:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 2/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 12 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.480961:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 4/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.481006:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 12 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.481022:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 12 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.481038:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371157632:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302842.481166:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e05480 going to sleep for 11 seconds 00000100:00100000:1.0:1789302842.483906:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.483964:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7dc00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371157632:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302842.483999:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.484073:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350461056 00000100:00100000:3.0:1789302842.484082:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350461056 00010000:00010000:3.0:1789302842.484128:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 13 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.484171:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 391 00000100:00100000:3.0:1789302842.484182:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:2348:x1876219350461056:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302842.484200:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302842.484209:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 4/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 13 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.484222:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 3/0,0 mode: PR/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 13 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.484249:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee2b lrc: 0/0,0 mode: --/PR res: [0x200000403:0x1:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b5a expref: 13 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.484266:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:3.0:1789302842.484272:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.484283:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.484296:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.484328:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302842.484398:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+12:2348:x1876219350461056:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 217us (506us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.484407:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 391 00010000:00010000:2.0:1789302842.485111:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485124:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485170:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485178:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485185:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485192:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.485197:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485206:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 4/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485218:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:2.0:1789302842.485221:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:2.0:1789302842.485226:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 2/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.485233:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ee7f lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.485307:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350460928:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 4470us (4929us total) trans 0 rc -22/-22 00000100:00100000:2.0:1789302842.485316:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1318 00000100:00100000:1.0:1789302842.486989:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.487091:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350461184 00000100:00100000:3.0:1789302842.487104:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350461184 00000100:00100000:3.0:1789302842.487175:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 365 00000100:00100000:3.0:1789302842.487187:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+7:2348:x1876219350461184:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302842.487242:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.487264:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.487274:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302842.487279:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302842.487284:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.487327:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 7 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.487344:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 7 pid: 6697 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.487362:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371157760:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302842.487542:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf3c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302842.491442:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.491627:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350461312 00000100:00100000:2.0:1789302842.491639:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350461312 00010000:00010000:2.0:1789302842.491696:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 8 pid: 6697 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.491738:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 392 00000100:00100000:2.0:1789302842.491749:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:2347:x1876219350461312:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302842.491775:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302842.491803:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 8 pid: 6697 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.491819:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 8 pid: 6697 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.491858:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302842.491864:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302842.491871:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.491888:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.491915:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302842.491973:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184ee6a lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->1048575) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b7d expref: 8 pid: 6697 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.491983:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+8:2347:x1876219350461312:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 234us (553us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.491993:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 392 00000100:00100000:1.0:1789302842.492098:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371157760:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302842.492136:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492155:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492174:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492205:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492214:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492223:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492231:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302842.492236:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492246:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492562:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.492579:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184ee86 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302842.492820:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2348:x1876219350461184:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 5633us (5838us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302842.492831:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 365 00000100:00100000:1.0:1789302842.494599:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.494680:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350461568 00000100:00100000:3.0:1789302842.494691:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350461568 00000100:00100000:3.0:1789302842.494751:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1319 00000100:00100000:3.0:1789302842.494774:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+11:30834:x1876219350461568:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302842.494884:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.494911:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.494925:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.495192:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.495204:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 2/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 12 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.495214:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 4/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.495417:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 3/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 12 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.495434:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 12 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.495447:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371157888:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302842.495687:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cfc00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302842.497953:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.498302:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350461696 00000100:00100000:0.0:1789302842.498315:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350461696 00010000:00010000:0.0:1789302842.498366:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 13 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.498423:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 393 00000100:00100000:0.0:1789302842.498436:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:2347:x1876219350461696:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302842.498456:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.498464:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 5/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 13 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.498479:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 4/0,0 mode: CR/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 13 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.498539:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302842.498547:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,1 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.498558:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.498566:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 4/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.498590:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302842.498657:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+13:2347:x1876219350461696:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 223us (716us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.498664:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 393 00010000:00010000:3.0:1789302842.499810:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184ee78 lrc: 0/0,0 mode: --/CR res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b84 expref: 12 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.499885:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371157888:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302842.499912:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.499923:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.499930:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302842.500156:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968171 00000001:00080000:3.0:1789302842.500380:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968172 00000001:00080000:3.0:1789302842.503898:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968173 00000100:00100000:3.0:1789302842.504097:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371158016:0@lo:1000:mdt00_001.0 00000100:00100000:3.0:1789302842.504190:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302842.504352:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cf900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302842.504651:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371158016 00000100:00100000:2.0:1789302842.504671:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371158016 00000100:00100000:2.0:1789302842.504739:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 63 00000100:00100000:2.0:1789302842.504751:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371158016:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789302842.504878:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371158016:12345-0@lo:1000:mdt00_001.0 Request processed in 129us (688us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.504886:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 63 00000100:00100000:0.0:1789302842.505179:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371158016:0@lo:1000:mdt00_001.0 00000001:00080000:0.0:1789302842.506144:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968174 00000100:00100000:0.0:1789302842.506253:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371158144:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302842.506751:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302842.506808:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302842.507173:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371158144 00000100:00100000:2.0:1789302842.507182:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371158144 00000100:00100000:2.0:1789302842.507277:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 64 00000100:00100000:2.0:1789302842.507289:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f07781c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371158144:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789302842.507629:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f07781c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5844:x1876219371158144:12345-0@lo:1000:mdt00_001.0 Request processed in 341us (877us total) trans 4294967372 rc 0/0 00000100:00100000:2.0:1789302842.507637:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 64 00000100:00100000:0.0:1789302842.508393:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371158144:0@lo:1000:mdt00_001.0 00000001:00080000:0.0:1789302842.508608:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968175 00010000:00010000:0.0:1789302842.508718:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,1 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508730:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 2/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508739:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508745:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.508750:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 3/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508768:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 4/0,0 mode: EX/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508788:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2800) 00010000:00010000:0.0:1789302842.508791:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302842.508796:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 2/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508803:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184ee94 lrc: 0/0,0 mode: --/EX res: [0x200000403:0x1:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508894:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508906:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508912:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508917:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.508920:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508925:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508933:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.508935:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302842.508943:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b29 expref: 98 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508956:0:5844:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b29 expref: 98 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.508965:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b29 expref: 98 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.509012:0:5844:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1000/0x64fe43095184ed60 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b29 expref: 98 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.509028:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158272:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302842.509177:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a840 going to sleep for 11 seconds 00000100:00100000:1.0:1789302842.512049:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.512787:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158272:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302842.512828:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.512851:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ed59 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.512898:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.512915:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.512926:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 4/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.512973:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 46 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.512991:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 46 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.513007:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158400:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302842.513110:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a840 going to sleep for 11 seconds 00000100:00100000:2.0:1789302842.515378:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350461824 00000100:00100000:2.0:1789302842.515398:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350461824 00000100:00100000:2.0:1789302842.515480:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 266 00000100:00100000:2.0:1789302842.515492:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e8c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30806:x1876219350461824:12345-192.168.203.11@tcp:49:cat.0 00000100:00100000:2.0:1789302842.515708:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e8c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30806:x1876219350461824:12345-192.168.203.11@tcp:49:cat.0 Request processed in 218us (3675us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.515716:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 266 00000100:00100000:1.0:1789302842.517885:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.517971:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350461952 00000100:00100000:3.0:1789302842.517984:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350461952 00010000:00010000:3.0:1789302842.518050:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 47 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.518097:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 394 00000100:00100000:3.0:1789302842.518109:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350461952:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302842.518126:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302842.518133:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 47 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.518149:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 47 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.518177:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302842.518184:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,1 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.518197:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.518207:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.518230:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302842.518290:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350461952:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 183us (418us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.518298:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 394 00010000:00010000:0.0:1789302842.518449:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184ed83 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b30 expref: 46 pid: 18427 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.518469:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbfb80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158400:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302842.518496:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518538:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518546:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518561:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518569:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518581:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518587:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f42044000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.518592:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518600:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518609:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.518612:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302842.518616:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 2/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518622:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044000/0x64fe43095184ee9b lrc: 0/0,0 mode: --/EX res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518638:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518644:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518649:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518654:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.518656:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518662:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518670:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.518672:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302842.518675:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.518681:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ee8d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.518936:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350461568:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 24174us (24345us total) trans 4294968176 rc 0/0 00000100:00100000:0.0:1789302842.518946:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1319 00000020:00080000:0.0:1789302842.518950:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:2.0:1789302842.519125:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302842.519131:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302842.519648:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1789302842.519657:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00080000:1.0:1789302842.519848:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302842.522071:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.522122:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.522353:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350462080 00000100:00100000:2.0:1789302842.522366:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350462080 00000100:00100000:2.0:1789302842.522459:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00100000:2.0:1789302842.522472:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+12:30806:x1876219350462080:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302842.522495:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302842.522547:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b8b expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302842.522580:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1b:0x0] initial LVB size: 1048576, mtime: 0x6aa69831, atime: 0x6aa69831, ctime: 0x6aa69831, blocks: 0x803 00010000:00010000:2.0:1789302842.522608:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=679s v=(0 0) 00010000:00010000:2.0:1789302842.522613:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302842.522625:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 13 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.522642:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 13 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.522657:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3a00, rc 0) 00000100:00100000:2.0:1789302842.522724:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:30806:x1876219350462080:12345-192.168.203.11@tcp:101:cat.0 Request processed in 254us (667us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.522732:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 366 00000100:00080000:1.0:1789302842.522975:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302842.523110:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350462336 00000100:00100000:3.0:1789302842.523117:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350462336 00000100:00100000:3.0:1789302842.523164:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 267 00000100:00100000:3.0:1789302842.523173:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+16:30834:x1876219350462336:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:3.0:1789302842.524035:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000403:0x1:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1789302842.524157:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968177 00010000:00010000:3.0:1789302842.524214:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 3/0,1 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.524233:0:12976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.524246:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 3/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.524257:0:12976:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.524264:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 4/0,1 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.524271:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.524297:0:12976:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302842.524396:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:30834:x1876219350462336:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1222us (2270us total) trans 4294968178 rc 0/0 00000100:00100000:3.0:1789302842.524405:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 267 00000020:00080000:3.0:1789302842.524412:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:0.0:1789302842.524756:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302842.524774:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.524782:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302842.524787:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 2/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.524797:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 3/0,0 mode: PW/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.524812:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000403:0x1:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:0.0:1789302842.524815:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302842.524821:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 1/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.524827:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1400/0x64fe43095184eea9 lrc: 0/0,0 mode: --/PW res: [0x200000403:0x1:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.524846:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302842.524848:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302842.524850:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302842.529285:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.529379:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350462464 00000100:00100000:3.0:1789302842.529392:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350462464 00002000:00100000:3.0:1789302842.529467:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1b:0x0] (0->1048575) req@ffff960f41c93b80 x1876219350462464/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:418/0 lens 488/0 e 0 to 0 dl 1789302853 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302842.529479:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c93b80 with x1876219350462464 ext(0->1048575) 00002000:00010000:3.0:1789302842.529485:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41c93b80 00000100:00100000:3.0:1789302842.529560:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 116 00000100:00100000:3.0:1789302842.529572:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350462464:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:2.0:1789302842.545123:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350462464:12345-192.168.203.11@tcp:3:cat.0 Request processed in 15535us (15845us total) trans 0 rc 1048576/1048576 00000100:00100000:2.0:1789302842.545140:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 116 00002000:00100000:2.0:1789302842.545157:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1b:0x0] (0->1048575) req@ffff960f41c93b80 x1876219350462464/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:418/0 lens 488/440 e 0 to 0 dl 1789302853 ref 1 fl Complete:/600/0 rc 1048576/1048576 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302842.545169:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c93b80 with x1876219350462464 ext(0->1048575) 00002000:00010000:2.0:1789302842.545177:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41c93b80 00000100:00100000:1.0:1789302842.554364:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.554815:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350462592 00000100:00100000:3.0:1789302842.554829:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350462592 00002000:00100000:3.0:1789302842.554895:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1b:0x0] (1048576->1052671) req@ffff960f41c91f80 x1876219350462592/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:418/0 lens 488/0 e 0 to 0 dl 1789302853 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302842.554907:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c91f80 with x1876219350462592 ext(1048576->1052671) 00002000:00010000:3.0:1789302842.554914:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41c91f80 00000100:00100000:3.0:1789302842.554952:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 117 00000100:00100000:3.0:1789302842.554965:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350462592:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:3.0:1789302842.555796:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350462592:12345-192.168.203.11@tcp:3:cat.0 Request processed in 830us (1441us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.555807:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 117 00002000:00100000:3.0:1789302842.555816:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1b:0x0] (1048576->1052671) req@ffff960f41c91f80 x1876219350462592/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:418/0 lens 488/440 e 0 to 0 dl 1789302853 ref 1 fl Complete:/600/0 rc 0/0 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302842.555826:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41c91f80 with x1876219350462592 ext(1048576->1052671) 00002000:00010000:3.0:1789302842.555830:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41c91f80 00000100:00100000:0.0:1789302842.569280:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302842.569400:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350462848 00000100:00100000:1.0:1789302842.569414:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350462848 00000100:00100000:1.0:1789302842.569640:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 268 00000100:00100000:1.0:1789302842.569653:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+99:30806:x1876219350462848:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302842.569994:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30806:x1876219350462848:12345-192.168.203.11@tcp:35:cat.0 Request processed in 343us (728us total) trans 4294968179 rc 0/0 00000100:00100000:1.0:1789302842.570002:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 268 00000100:00100000:2.0:1789302842.610782:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.611024:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350462976 00000100:00100000:0.0:1789302842.611038:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350462976 00000100:00100000:0.0:1789302842.611122:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1320 00000100:00100000:0.0:1789302842.611134:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:30836:x1876219350462976:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:0.0:1789302842.611180:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.611211:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184eeb0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b92 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611312:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611330:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611344:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611361:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611369:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611376:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611805:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611816:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.611823:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302842.611877:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:0.0:1789302842.611883:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302842.612345:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612366:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612376:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612383:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612390:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612397:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.612402:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612410:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612424:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:0.0:1789302842.612428:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302842.612434:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612441:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeb7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612455:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612462:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612469:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612474:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.612477:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612483:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612491:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302842.612493:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302842.612496:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612617:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eebe lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612677:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184eeb0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 102 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612694:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 101 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.612702:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.612704:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302842.612707:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00000100:00100000:0.0:1789302842.612813:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:30836:x1876219350462976:12345-192.168.203.11@tcp:101:stat.0 Request processed in 1680us (2055us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.612826:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1320 00000100:00100000:2.0:1789302842.642282:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.642452:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350463104 00000100:00100000:0.0:1789302842.642466:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350463104 00000100:00100000:0.0:1789302842.642580:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1321 00000100:00100000:0.0:1789302842.642593:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30837:x1876219350463104:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:0.0:1789302842.642621:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.642647:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184eecc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0b99 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.642747:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.642774:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.642788:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.642805:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.642822:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.642828:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.643200:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.643215:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.643223:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302842.643876:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00000004:00200000:0.0:1789302842.643883:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302842.643972:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.643988:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.643999:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644007:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644014:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644021:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.644026:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644034:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644049:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:0.0:1789302842.644053:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302842.644059:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644066:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eed3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644082:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644088:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644094:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644158:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.644162:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644169:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644177:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302842.644180:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302842.644183:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644188:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeda lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644236:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184eecc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644253:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.644261:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.644324:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302842.644328:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00000100:00100000:0.0:1789302842.644453:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30837:x1876219350463104:12345-192.168.203.11@tcp:101:stat.0 Request processed in 1863us (2197us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.644462:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1321 00000100:00100000:1.0:1789302842.650012:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.652987:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350463232 00000100:00100000:0.0:1789302842.653008:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350463232 00000100:00100000:0.0:1789302842.653173:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 367 00000100:00100000:0.0:1789302842.653184:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350463232:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:0.0:1789302842.653214:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.653238:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ba0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.653270:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302842.653275:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302842.653285:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.653304:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.653325:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b600, rc 0) 00000100:00100000:0.0:1789302842.653422:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350463232:12345-192.168.203.11@tcp:101:stat.0 Request processed in 239us (3424us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.653430:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 367 00000100:00080000:1.0:1789302842.657638:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302842.668234:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.668387:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350463488 00000100:00100000:0.0:1789302842.668400:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350463488 00000100:00100000:0.0:1789302842.668484:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1322 00000100:00100000:0.0:1789302842.668498:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:30553:x1876219350463488:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302842.668575:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.668606:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeef lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ba7 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.668749:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.668776:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 102 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.668785:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.668793:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 5/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.669063:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 46 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.669084:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 46 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.669101:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158528:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302842.669222:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 102 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.669233:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 102 pid: 6919 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.669242:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158656:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302842.669395:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f73a840 going to sleep for 11 seconds 00000100:00100000:0.0:1789302842.686247:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbe300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158528:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302842.686278:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371158656:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302842.686316:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.688656:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.688692:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.688869:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350463744 00000100:00100000:0.0:1789302842.688880:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350463744 00010000:00010000:0.0:1789302842.688944:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 103 pid: 6919 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.688987:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350463616 00000100:00100000:0.0:1789302842.688992:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350463616 00010000:00010000:0.0:1789302842.689008:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 47 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.689043:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 395 00000100:00100000:0.0:1789302842.689055:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120b9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2348:x1876219350463744:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302842.689074:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.689082:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 103 pid: 6919 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689096:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 103 pid: 6919 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689120:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184eec5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b92 expref: 103 pid: 6919 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689140:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.689145:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689155:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 2 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689162:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302842.689238:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120b9850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:2348:x1876219350463744:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 185us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.689246:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 395 00000100:00100000:0.0:1789302842.689597:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 396 00000100:00100000:0.0:1789302842.689606:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350463616:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302842.689619:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.689625:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 47 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689636:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 47 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689650:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184eee1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b99 expref: 47 pid: 5843 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689660:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.689663:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689671:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689681:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 4/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689707:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302842.689733:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2348:x1876219350463616:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 128us (1041us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.689739:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 396 00010000:00010000:0.0:1789302842.689788:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.689797:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,1 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302842.689825:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 1048576 00010000:00010000:0.0:1789302842.690542:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184eeef lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 103 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.690564:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 102 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.690572:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.690575:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:0.0:1789302842.690578:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a800, rc 0) 00000100:00100000:0.0:1789302842.690621:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30553:x1876219350463488:12345-192.168.203.11@tcp:101:bash.0 Request processed in 22127us (22402us total) trans 4294968180 rc 0/0 00000100:00100000:0.0:1789302842.690629:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1322 00000100:00100000:2.0:1789302842.696776:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.696992:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350464000 00000100:00100000:0.0:1789302842.697006:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350464000 00000100:00100000:0.0:1789302842.697114:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1323 00000100:00100000:0.0:1789302842.697126:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:30553:x1876219350464000:12345-192.168.203.11@tcp:36:bash.0 00010000:00010000:0.0:1789302842.697550:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.697575:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.697589:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302842.697651:0:6919:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968181 00000004:00010000:0.0:1789302842.697681:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000080daa442 reply state 0000000024d8cc7e transno 4294968182 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.697692:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000001:00080000:0.0:1789302842.697744:0:6919:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968182 00000100:00100000:0.0:1789302842.697897:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:30553:x1876219350464000:12345-192.168.203.11@tcp:36:bash.0 Request processed in 771us (1142us total) trans 4294968183 rc 0/0 00000100:00100000:0.0:1789302842.700805:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1323 00000020:00080000:0.0:1789302842.700825:0:6919:0:(tgt_main.c:784:tgt_ses_key_exit()) total 3 transactions per RPC 00000100:00100000:1.0:1789302842.701438:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302842.701642:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.701660:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.701669:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.701676:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302842.701683:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.701692:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.701713:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:1.0:1789302842.701717:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302842.701723:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302842.701729:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184ef04 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x27:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.701833:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350464128 00000100:00100000:3.0:1789302842.701847:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350464128 00000100:00100000:3.0:1789302842.701919:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 368 00000100:00100000:3.0:1789302842.701930:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:30553:x1876219350464128:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302842.701951:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302842.701974:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a32dbd22/0x64fe43095184ef0b lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bae expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.702012:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 14 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.702023:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 9 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.702032:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302842.702037:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302842.702042:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32dbd22/0x64fe43095184ef0b lrc: 5/0,0 mode: --/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000080000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bae expref: 14 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.702085:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 9 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.702101:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 9 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.702118:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371158784:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302842.702240:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 14 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.702251:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 14 pid: 13439 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.702267:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371158912:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302842.702294:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f258cff00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302842.712941:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.712982:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.713124:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350464256 00000100:00100000:0.0:1789302842.713137:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350464256 00010000:00010000:0.0:1789302842.713191:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 10 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.713249:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350464384 00000100:00100000:0.0:1789302842.713253:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350464384 00010000:00010000:0.0:1789302842.713280:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 15 pid: 13439 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.713305:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 397 00000100:00100000:0.0:1789302842.713315:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350464256:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302842.713334:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.713341:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 10 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.713354:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 10 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.713393:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302842.713467:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350464256:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 152us (540us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.713475:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 397 00000100:00100000:0.0:1789302842.713497:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 398 00000100:00100000:0.0:1789302842.713531:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2348:x1876219350464384:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302842.713541:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302842.713546:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 15 pid: 13439 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.713557:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 15 pid: 13439 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.713573:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302842.713580:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302842.713589:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000a32dbd22/0x64fe43095184ef0b lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080080000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bae expref: 15 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.713602:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a32dbd22/0x64fe43095184ef0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50080080000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bae expref: 15 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.713647:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000a32dbd22/0x64fe43095184ef0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bae expref: 15 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.713670:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371159040:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302842.713732:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f01254e40 going to sleep for 11 seconds 00010000:00010000:3.0:1789302842.714629:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000042078c59/0x64fe43095184eee8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba0 expref: 9 pid: 18578 timeout: 779 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.714655:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371158784:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302842.714670:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000edc87b38/0x64fe43095184eea2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b8b expref: 15 pid: 13439 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.714679:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371158912:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302842.714723:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a32dbd22/0x64fe43095184ef0b lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000080020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bae expref: 14 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.714735:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1400, rc 0) 00000100:00100000:3.0:1789302842.714788:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:30553:x1876219350464128:12345-192.168.203.11@tcp:101:bash.0 Request processed in 12860us (13356us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.714796:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 368 00000100:00100000:0.0:1789302842.720049:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371159040:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302842.720083:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302842.720160:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350464384:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 6630us (7175us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.720173:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 398 00000100:00100000:2.0:1789302842.720452:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.720562:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350464512 00000100:00100000:0.0:1789302842.720573:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350464512 00002000:00010000:0.0:1789302842.720619:0:18429:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/27 (0->18446744073709551615) 00002000:00100000:0.0:1789302842.720624:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7f100 with x1876219350464512 ext(0->18446744073709551615) 00002000:00010000:0.0:1789302842.720638:0:18429:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ef7f100. 00000100:00100000:0.0:1789302842.720666:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 118 00000100:00100000:0.0:1789302842.720677:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2347:x1876219350464512:12345-192.168.203.11@tcp:10:bash.0 00002000:00010000:0.0:1789302842.721249:0:18429:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb size from disk: 1048576 -> 0 00002000:00010000:0.0:1789302842.721259:0:18429:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb mtime from disk: 1789302833 -> 1789302842 00002000:00010000:0.0:1789302842.721261:0:18429:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb atime from disk: 1789302833 -> 1789302833 00002000:00010000:0.0:1789302842.721263:0:18429:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb ctime from disk: 1789302833 -> 1789302842 00002000:00010000:0.0:1789302842.721265:0:18429:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1b:0x0] updating lvb blocks from disk: 2051 -> 2051 00000100:00100000:0.0:1789302842.721334:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2347:x1876219350464512:12345-192.168.203.11@tcp:10:bash.0 Request processed in 654us (1436us total) trans 4294967373 rc 0/0 00000100:00100000:0.0:1789302842.721346:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 118 00002000:00010000:0.0:1789302842.721352:0:18429:0:(ofd_dev.c:2827:ofd_punch_hpreq_check()) lustre-OST0000: refresh locks: 9663677440/27 (0->18446744073709551615) 00002000:00100000:0.0:1789302842.721357:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef7f100 with x1876219350464512 ext(0->18446744073709551615) 00002000:00010000:0.0:1789302842.721364:0:18429:0:(ofd_dev.c:2832:ofd_punch_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3ef7f100. 00000100:00100000:1.0:1789302842.741631:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.741813:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350464640 00000100:00100000:0.0:1789302842.741830:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350464640 00000100:00100000:0.0:1789302842.741907:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1324 00000100:00100000:0.0:1789302842.741918:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:30838:x1876219350464640:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:0.0:1789302842.741947:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.741971:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184ef12 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bb5 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742039:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742055:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742066:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742080:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742088:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742094:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742379:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742395:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 2/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 106 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742403:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 4/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x48 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742454:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 106 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.742472:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 106 pid: 5844 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302842.742488:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371159168:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302842.742615:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f012549c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302842.746055:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.746376:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371159168:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302842.746413:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x48 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.746489:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350464896 00000100:00100000:2.0:1789302842.746535:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350464896 00000100:00100000:2.0:1789302842.746604:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 269 00000100:00100000:2.0:1789302842.746619:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:5586:x1876219350464896:12345-192.168.203.11@tcp:35:ldlm_bl.0 00000100:00100000:2.0:1789302842.752672:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:5586:x1876219350464896:12345-192.168.203.11@tcp:35:ldlm_bl.0 Request processed in 6047us (6619us total) trans 4294968184 rc 0/0 00000100:00100000:2.0:1789302842.752685:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 269 00000100:00100000:2.0:1789302842.754982:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302842.755369:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465024 00000100:00100000:3.0:1789302842.755379:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465024 00010000:00010000:3.0:1789302842.755427:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 108 pid: 5844 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302842.755469:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 399 00000100:00100000:3.0:1789302842.755479:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350465024:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302842.755494:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302842.755533:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 4/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 108 pid: 5844 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.755548:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 3/0,0 mode: CW/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 108 pid: 5844 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.755568:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184eef6 lrc: 0/0,0 mode: --/CW res: [0x200000402:0x27:0x0].0x0 bits 0x5/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ba7 expref: 108 pid: 5844 timeout: 779 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.755599:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:3.0:1789302842.755605:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x13/0x48 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.755615:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.755623:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 4/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302842.755659:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302842.755705:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:2348:x1876219350465024:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 227us (730us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.755713:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 399 00010000:00010000:2.0:1789302842.755997:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756007:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302842.756052:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 0 00000004:00200000:2.0:1789302842.756058:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302842.756080:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756093:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756103:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756112:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756120:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756127:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.756131:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756140:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756153:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:2.0:1789302842.756156:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.756162:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756168:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184ef19 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756184:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756192:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756198:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756204:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302842.756207:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756213:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756220:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302842.756223:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302842.756226:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756232:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184ef20 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756271:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184ef12 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bb5 expref: 107 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756285:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a000/0x64fe43095184ef27 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bb5 expref: 106 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.756293:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:2.0:1789302842.756295:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302842.756298:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a000, rc 0) 00000100:00100000:2.0:1789302842.756339:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:30838:x1876219350464640:12345-192.168.203.11@tcp:101:stat.0 Request processed in 14423us (14723us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302842.756347:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1324 00000100:00100000:2.0:1789302842.775709:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302842.775856:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465152 00000100:00100000:0.0:1789302842.775866:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465152 00000100:00100000:0.0:1789302842.775931:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1325 00000100:00100000:0.0:1789302842.775942:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:30839:x1876219350465152:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:0.0:1789302842.775961:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302842.775981:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184ef2e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bbc expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776054:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776068:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776080:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776094:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776101:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776107:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776398:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b200/0x64fe43095184ef43 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x27:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776408:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b200/0x64fe43095184ef43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776415:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b200/0x64fe43095184ef43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302842.776450:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x27:0x0]: returning size 0 00000004:00200000:0.0:1789302842.776454:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302842.776471:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b200/0x64fe43095184ef43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776487:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b200/0x64fe43095184ef43 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776495:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776539:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776548:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776554:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.776558:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776565:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776576:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:0.0:1789302842.776579:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302842.776585:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776592:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ef35 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776604:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776610:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776616:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776622:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302842.776624:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776630:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776638:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0xa624d530 (ffff960f09ae1000) 00010000:00010000:0.0:1789302842.776640:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302842.776643:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776650:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ef3c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0xa624d530 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776680:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184ef2e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4b:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bbc expref: 48 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776694:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b200/0x64fe43095184ef43 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x27:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bbc expref: 47 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302842.776702:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x27:0x0].0x0 (ffff960f0cdf2000) 00010000:00010000:0.0:1789302842.776704:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302842.776707:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b200, rc 0) 00000100:00100000:0.0:1789302842.778816:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:30839:x1876219350465152:12345-192.168.203.11@tcp:101:stat.0 Request processed in 2862us (3102us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302842.778832:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1325 00000100:00100000:1.0:1789302842.785996:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302842.786083:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465280 00000100:00100000:2.0:1789302842.786094:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465280 00000100:00100000:2.0:1789302842.786152:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 369 00000100:00100000:2.0:1789302842.786163:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350465280:12345-192.168.203.11@tcp:101:stat.0 00010000:00010000:2.0:1789302842.786182:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302842.786200:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a010bee/0x64fe43095184ef4a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bc3 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302842.786268:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371159296:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302842.786366:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f10e053c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302842.794047:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371159296:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302842.794087:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a010bee/0x64fe43095184ef4a lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bc3 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.794112:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a010bee/0x64fe43095184ef4a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bc3 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302842.794126:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:2.0:1789302842.794185:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350465280:12345-192.168.203.11@tcp:101:stat.0 Request processed in 8023us (8200us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302842.794194:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 369 00000100:00100000:2.0:1789302842.893111:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371159424:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302842.893185:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302842.893345:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371159424 00000100:00100000:3.0:1789302842.893353:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371159424 00000100:00100000:3.0:1789302842.893407:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:3.0:1789302842.893416:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371159424:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302842.893453:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302842.893524:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371159424:12345-0@lo:13:osp-pre-1-0.0 Request processed in 109us (338us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302842.893542:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1789302842.893686:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371159424:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302843.021772:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302843.404917:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371159552:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302843.405017:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302843.405168:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371159552 00000100:00100000:3.0:1789302843.405180:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371159552 00000100:00100000:3.0:1789302843.405272:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 201 00000100:00100000:3.0:1789302843.405286:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371159552:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302843.405318:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3858759680 avail 3855613952 00000100:00100000:3.0:1789302843.411680:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371159552:12345-0@lo:13:osp-pre-0-0.0 Request processed in 6380us (6645us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302843.411700:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 201 00000100:00100000:0.0:1789302843.411916:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371159552:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302843.672938:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302843.672984:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302843.673025:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302843.673054:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302843.673074:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302843.673112:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302843.673150:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465664 00000100:00100000:3.0:1789302843.673160:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465664 00000100:00100000:1.0:1789302843.673163:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465920 00000100:00100000:1.0:1789302843.673180:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465920 00000100:00100000:3.0:1789302843.673233:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465536 00000100:00100000:3.0:1789302843.673238:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465536 00000100:00100000:1.0:1789302843.673248:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350466048 00000100:00100000:1.0:1789302843.673255:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350466048 00000100:00100000:3.0:1789302843.673273:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 480 00000100:00100000:3.0:1789302843.673289:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350465664:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302843.673303:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 481 00000100:00100000:1.0:1789302843.673318:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2345:x1876219350465920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302843.673370:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 482 00000100:00100000:3.0:1789302843.673371:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350465664:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 86us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302843.673383:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 480 00000100:00100000:0.0:1789302843.673390:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350465536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302843.673420:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 483 00000100:00100000:1.0:1789302843.673420:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+6:2345:x1876219350465920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 98us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302843.673484:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465408 00000100:00100000:2.0:1789302843.673491:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465408 00000100:00100000:3.0:1789302843.675333:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2345:x1876219350466048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302843.675333:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350465536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 59us (384us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302843.675390:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 604 00000100:00100000:2.0:1789302843.675407:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:2345:x1876219350465408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302843.675421:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2345:x1876219350466048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1986us (2305us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302843.675443:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:2345:x1876219350465408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 41us (2513us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302843.675451:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 604 00000100:00080000:2.0:1789302843.675537:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302843.678320:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 481 00000100:00100000:2.0:1789302843.678338:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350465792 00000100:00100000:2.0:1789302843.678348:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350465792 00000100:00100000:2.0:1789302843.678535:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 469 00000100:00100000:0.0:1789302843.680582:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 482 00000100:00080000:1.0:1789302843.681802:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302843.681998:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 483 00000100:00100000:2.0:1789302843.682961:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:2345:x1876219350465792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302843.683142:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+18:2345:x1876219350465792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 202us (10065us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302843.685228:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 469 00010000:00010000:1.0:1789302844.044762:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302845.069320:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302846.094663:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302847.116665:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302847.500918:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c47480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371159680:0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.501035:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302847.501098:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371160192:0@lo:400:kworker.0 00000100:00100000:0.0:1789302847.503379:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371160064:0@lo:400:kworker.0 00000100:00100000:2.0:1789302847.507075:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371159936:0@lo:400:kworker.0 00000100:00100000:2.0:1789302847.512009:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302847.514583:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302847.514641:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1789302847.514654:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371159808:0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.514713:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302847.514745:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371160320:0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.514773:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302847.514831:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371159680 00000100:00100000:3.0:1789302847.514840:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371159680 00000100:00100000:3.0:1789302847.514889:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:3.0:1789302847.514898:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371159680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.514954:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371159680:12345-0@lo:400:kworker.0 Request processed in 57us (13929us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302847.514961:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00100000:3.0:1789302847.514995:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160320 00000100:00100000:3.0:1789302847.514999:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160320 00000100:00100000:3.0:1789302847.515038:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789302847.515045:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371160320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.515078:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371160320:12345-0@lo:400:kworker.0 Request processed in 34us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302847.515086:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00080000:3.0:1789302847.515112:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302847.515186:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371160320:0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.515206:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c47480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371159680:0@lo:400:kworker.0 00000100:00100000:0.0:1789302847.515262:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371159936 00000100:00100000:0.0:1789302847.515276:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371159936 00000100:00100000:0.0:1789302847.515347:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160192 00000100:00100000:0.0:1789302847.515351:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160192 00000100:00100000:0.0:1789302847.515374:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371159808 00000100:00100000:0.0:1789302847.515378:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371159808 00000100:00100000:0.0:1789302847.515396:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:0.0:1789302847.516362:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371159936:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789302847.518646:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302847.519846:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371159936:12345-0@lo:400:kworker.0 Request processed in 3487us (7829us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302847.519865:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1789302847.520231:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1789302847.520248:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371160192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302847.520342:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371160192:12345-0@lo:400:kworker.0 Request processed in 100us (5845us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302847.520350:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1789302847.520378:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:2.0:1789302847.520386:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371159808:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302847.520417:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371159808:12345-0@lo:400:kworker.0 Request processed in 32us (5704us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302847.520423:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:2.0:1789302847.520472:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371159936:0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.522194:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371159808:0@lo:400:kworker.0 00000100:00100000:3.0:1789302847.522214:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371160192:0@lo:400:kworker.0 00000100:00100000:2.0:1789302847.524131:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302847.526663:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160064 00000100:00100000:1.0:1789302847.526678:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160064 00000100:00100000:1.0:1789302847.526775:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:1.0:1789302847.526787:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371160064:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789302847.526848:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302847.526881:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371160064:12345-0@lo:400:kworker.0 Request processed in 97us (2752us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302847.526889:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:2.0:1789302847.526905:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c47800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371160064:0@lo:400:kworker.0 00000001:00080000:1.0:1789302847.609417:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968183 is committed 00000001:00080000:1.0:1789302847.609440:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968176 is committed 00000020:00080000:1.0:1789302847.609448:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f3f73a780 refcount 2 committed 0 result 0 batchid 30 00000020:00080000:1.0:1789302847.609453:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3b0b4600 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f3f73ad80 00000020:00080000:1.0:1789302847.609456:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3b0b4400 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3fa19800 00000001:00080000:1.0:1789302847.609477:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968184 is committed 00000001:00080000:1.0:1789302847.609486:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968178 is committed 00000020:00080000:1.0:1789302847.609534:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 30 committed batchid 29 00000020:00080000:1.0:1789302847.609706:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 29 00000001:00080000:2.0:1789302847.621825:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967373 is committed 00000001:00080000:2.0:1789302847.621848:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967372 is committed 00000001:00080000:2.0:1789302847.621856:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967371 is committed 00000100:00100000:2.0:1789302848.013375:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371160448:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302848.013446:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302848.013734:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160448 00000100:00100000:3.0:1789302848.013744:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160448 00000100:00100000:3.0:1789302848.013810:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:3.0:1789302848.013820:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371160448:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302848.013857:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302848.013920:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371160448:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (473us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302848.013927:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1789302848.014020:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371160448:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302848.070229:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302848.070349:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350466176 00000100:00100000:0.0:1789302848.070359:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350466176 00000100:00100000:0.0:1789302848.070441:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1326 00000100:00100000:0.0:1789302848.070453:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30833:x1876219350466176:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1789302848.070589:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30833:x1876219350466176:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 139us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302848.070597:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1326 00000100:00080000:2.0:1789302848.070712:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302848.077695:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.079291:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350466432 00000100:00100000:3.0:1789302848.079305:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350466432 00000100:00100000:3.0:1789302848.079381:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 270 00000100:00100000:3.0:1789302848.079391:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:30833:x1876219350466432:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302848.079710:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:30833:x1876219350466432:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 319us (2025us total) trans 4294968185 rc 0/0 00000100:00100000:3.0:1789302848.079717:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 270 00010000:00010000:0.0:1789302848.140754:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302848.524911:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302848.524945:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371160576:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302848.525024:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302848.525133:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160576 00000100:00100000:3.0:1789302848.525151:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160576 00000100:00100000:3.0:1789302848.525236:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 203 00000100:00100000:3.0:1789302848.525249:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371160576:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302848.525316:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302848.525426:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371160576:12345-0@lo:13:osp-pre-0-0.0 Request processed in 180us (401us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302848.525444:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 203 00000100:00100000:0.0:1789302848.525584:0:3302:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967372 gen 2 00000004:00080000:0.0:1789302848.525593:0:3302:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967372 committed transno 4294967372 00000100:00100000:0.0:1789302848.525628:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371160576:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302848.787880:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.788013:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.788058:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.788089:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.788114:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.788134:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.788157:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302848.788183:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302848.788254:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467072 00000100:00100000:1.0:1789302848.788270:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467072 00000100:00100000:3.0:1789302848.788331:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302848.788364:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350466688 00000100:00100000:1.0:1789302848.788376:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350466688 00000100:00100000:3.0:1789302848.788384:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302848.788435:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467456 00000100:00100000:1.0:1789302848.788441:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467456 00000100:00100000:1.0:1789302848.788462:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00100000:1.0:1789302848.788471:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350467072:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302848.788593:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2348:x1876219350467072:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 122us (534us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302848.788601:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00100000:1.0:1789302848.788618:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 474 00000100:00100000:1.0:1789302848.788625:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:2348:x1876219350466688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302848.788655:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:2348:x1876219350466688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302848.788661:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 474 00000100:00100000:1.0:1789302848.788672:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 475 00000100:00100000:1.0:1789302848.788678:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2348:x1876219350467456:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302848.788723:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2348:x1876219350467456:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302848.788730:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 475 00000100:00080000:1.0:1789302848.788753:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302848.791277:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350466560 00000100:00100000:2.0:1789302848.791295:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350466560 00000100:00100000:2.0:1789302848.791632:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 606 00000100:00100000:2.0:1789302848.791726:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:2348:x1876219350466560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302848.791823:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:2348:x1876219350466560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 111us (3963us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302848.791831:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 606 00000100:00100000:0.0:1789302848.792928:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350466816 00000100:00100000:0.0:1789302848.792942:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350466816 00000100:00100000:0.0:1789302848.793017:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467328 00000100:00100000:0.0:1789302848.793021:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467328 00000100:00100000:0.0:1789302848.793037:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467584 00000100:00100000:0.0:1789302848.793040:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467584 00000100:00100000:0.0:1789302848.793053:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350466944 00000100:00100000:0.0:1789302848.793056:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350466944 00000100:00100000:0.0:1789302848.793067:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467200 00000100:00100000:0.0:1789302848.793069:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467200 00000100:00100000:0.0:1789302848.793082:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467712 00000100:00100000:0.0:1789302848.793085:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467712 00000100:00100000:0.0:1789302848.793106:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 486 00000100:00100000:0.0:1789302848.793117:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350466816:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302848.793185:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00100000:3.0:1789302848.793209:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350467328:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302848.793211:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350466816:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 87us (5188us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302848.793230:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 486 00000100:00100000:0.0:1789302848.793270:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:3.0:1789302848.793286:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350467328:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 83us (5190us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302848.793287:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350467584:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302848.793311:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00100000:0.0:1789302848.793341:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350467584:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 60us (5223us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302848.793347:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:0.0:1789302848.793362:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:3.0:1789302848.793379:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350466944:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302848.793417:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350466944:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 56us (5259us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302848.793425:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:3.0:1789302848.793439:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 490 00000100:00100000:3.0:1789302848.793590:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350467200:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302848.793620:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350467200:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 175us (5437us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302848.793627:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 490 00000100:00100000:3.0:1789302848.793641:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00100000:3.0:1789302848.793649:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350467712:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302848.793675:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2348:x1876219350467712:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (5293us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302848.793681:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00080000:3.0:1789302848.794868:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302848.794876:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789302849.165084:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302850.188695:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302850.701201:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302850.701213:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302850.701495:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 2 00000040:00080000:1.0:1789302850.701523:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302851.213809:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302851.442804:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302851.442850:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302851.442888:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302851.442956:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467840 00000100:00100000:3.0:1789302851.442978:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350468096 00000100:00100000:3.0:1789302851.442986:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350468096 00000100:00100000:3.0:1789302851.443056:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350467968 00000100:00100000:3.0:1789302851.443059:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467968 00000100:00100000:0.0:1789302851.443077:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350467840 00000100:00100000:3.0:1789302851.443094:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 59 00000100:00100000:3.0:1789302851.443106:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:66f3b492-6139-4ca4-a2a9-05f6a44cca16+18:2345:x1876219350468096:12345-192.168.203.11@tcp:103:ldlm_bl.0 00000100:00100000:0.0:1789302851.443114:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 60 00010000:00010000:3.0:1789302851.443124:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1789302851.443125:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120b8450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:66f3b492-6139-4ca4-a2a9-05f6a44cca16+18:2345:x1876219350467968:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302851.443137:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302851.443153:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f07ef6600/0x64fe43095184eb22 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 6 type: PLN flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a49 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302851.443154:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960e028b2200/0x64fe43095184eb29 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 8 type: PLN flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a50 expref: 18 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302851.443186:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302851.443186:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302851.443245:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120b8450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:66f3b492-6139-4ca4-a2a9-05f6a44cca16+16:2345:x1876219350467968:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 122us (354us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302851.443253:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 60 00000100:00100000:2.0:1789302851.448748:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 61 00000100:00100000:2.0:1789302851.448777:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+15:2345:x1876219350467840:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302851.448797:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302851.448821:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: ffff960f3fcf0200/0x64fe43095184eb1b lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a1f expref: 15 pid: 5836 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302851.448839:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302851.448928:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:66f3b492-6139-4ca4-a2a9-05f6a44cca16+14:2345:x1876219350468096:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 5823us (6075us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302851.448938:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 59 00000100:00100000:3.0:1789302851.449071:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302851.449158:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350468224 00000100:00100000:3.0:1789302851.449166:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350468224 00000100:00100000:3.0:1789302851.449210:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 370 00000100:00100000:3.0:1789302851.449217:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2344:x1876219350468224:12345-192.168.203.11@tcp:9:umount.0 00000020:00000080:3.0:1789302851.449242:0:13439:0:(genops.c:1467:class_disconnect()) disconnect: cookie 0x64fe43095184eb3e 00000100:00080000:3.0:1789302851.449251:0:13439:0:(import.c:65:import_set_state_nolock()) ffff960f1d83c000 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1789302851.449287:0:13439:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294967373 00000100:00100000:3.0:1789302851.449806:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+2:2344:x1876219350468224:12345-192.168.203.11@tcp:9:umount.0 Request processed in 587us (733us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302851.449815:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 370 00010000:00010000:3.0:1789302851.449838:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789302851.449846:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302851.453648:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350467840:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4872us (10840us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302851.453663:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 61 00000100:00100000:3.0:1789302851.464105:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302851.464282:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350468352 00000100:00100000:1.0:1789302851.464293:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350468352 00000100:00100000:1.0:1789302851.464367:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 371 00000100:00100000:1.0:1789302851.464378:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+5:2344:x1876219350468352:12345-192.168.203.11@tcp:9:umount.0 00000020:00000080:1.0:1789302851.464407:0:6695:0:(genops.c:1467:class_disconnect()) disconnect: cookie 0x64fe43095184eb37 00000100:00080000:1.0:1789302851.464417:0:6695:0:(import.c:65:import_set_state_nolock()) ffff960f10084800 : changing import state from FULL to CLOSED 00000100:00080000:3.0:1789302851.464523:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00000004:1.0:1789302851.464626:0:6695:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294967373 00000100:00100000:1.0:1789302851.464906:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+2:2344:x1876219350468352:12345-192.168.203.11@tcp:9:umount.0 Request processed in 528us (808us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302851.464916:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 371 00010000:00010000:0.0:1789302851.465115:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302851.475138:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302851.475306:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350468992 00000100:00100000:1.0:1789302851.475316:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350468992 00000100:00100000:1.0:1789302851.475412:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1327 00000100:00100000:1.0:1789302851.475425:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+10:2344:x1876219350468992:12345-192.168.203.11@tcp:39:umount.0 00000020:00000080:1.0:1789302851.475457:0:5844:0:(genops.c:1467:class_disconnect()) disconnect: cookie 0x64fe43095184eb30 00000020:00010000:1.0:1789302851.475475:0:5844:0:(genops.c:1920:obd_stale_export_put()) Put export ffff960f10081800: total 0 00000100:00080000:1.0:1789302851.475480:0:5844:0:(import.c:65:import_set_state_nolock()) ffff960f10083000 : changing import state from FULL to CLOSED 00000001:00000004:1.0:1789302851.475543:0:5844:0:(tgt_lastrcvd.c:727:tgt_server_data_update()) lustre-MDT0000_UUID: mount_count is 1, last_transno is 4294968185 00000020:00010000:0.0:1789302851.475713:0:17746:0:(genops.c:1906:obd_stale_export_get()) Get export ffff960f10081800: total 0 00010000:00010000:0.0:1789302851.475764:0:17746:0:(ldlm_lock.c:2651:ldlm_export_cancel_blocked_locks()) Export ffff960f10081800, canceled 0 locks, left on hash table 4. 00010000:00010000:0.0:1789302851.475843:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:0.0:1789302851.475847:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302851.475856:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42044600/0x64fe43095184eb68 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a81 expref: 9 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302851.475862:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:36d456e5-2a1b-4d58-a7f1-78ab3c35acb0+9:2344:x1876219350468992:12345-192.168.203.11@tcp:39:umount.0 Request processed in 441us (733us total) trans 4294968186 rc 0/0 00000100:00100000:1.0:1789302851.475870:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1327 00010000:00010000:0.0:1789302851.475891:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4b:0x0].0x0 (ffff960f2c94db00) 00010000:00010000:0.0:1789302851.475895:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302851.475899:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184edfa lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4b:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b4c expref: 7 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302851.475912:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:0.0:1789302851.475914:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302851.475918:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184ed91 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0b37 expref: 6 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302851.475928:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302851.475931:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302851.475934:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184eb84 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a88 expref: 5 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302851.475944:0:17746:0:(ldlm_lock.c:2679:ldlm_export_cancel_locks()) Export ffff960f10081800, canceled 4 locks, left on hash table 0. 00010000:00010000:0.0:1789302851.475957:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302851.475959:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302851.475961:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1789302851.476102:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302852.236795:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302852.625854:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f045500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371160960:0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.625922:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302852.625982:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302852.625989:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371161088:0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.626013:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302852.626046:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160960 00000100:00100000:2.0:1789302852.626053:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160960 00000100:00100000:2.0:1789302852.626104:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1789302852.626114:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371160960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.626161:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371160960:12345-0@lo:400:kworker.0 Request processed in 49us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302852.626168:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1789302852.626208:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f045500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371160960:0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.626232:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371161088 00000100:00100000:2.0:1789302852.626235:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371161088 00000100:00100000:2.0:1789302852.626266:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1789302852.626274:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371161088:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302852.626902:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f047100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371160704:0@lo:400:kworker.0 00000100:00100000:3.0:1789302852.626941:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302852.626971:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371161216:0@lo:400:kworker.0 00000100:00100000:3.0:1789302852.626993:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302852.627014:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1789302852.627020:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371160832:0@lo:400:kworker.0 00000100:00100000:3.0:1789302852.627045:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302852.627063:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371161344:0@lo:400:kworker.0 00000100:00100000:3.0:1789302852.627082:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789302852.627110:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302852.627115:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302852.627125:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160704 00000100:00100000:3.0:1789302852.627130:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160704 00000100:00100000:3.0:1789302852.627159:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 607 00000100:00100000:3.0:1789302852.627166:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371160704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302852.627200:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371160704:12345-0@lo:400:kworker.0 Request processed in 34us (259us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302852.627206:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 607 00000100:00100000:3.0:1789302852.627232:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371161216 00000100:00100000:3.0:1789302852.627235:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371161216 00000100:00100000:3.0:1789302852.627261:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371160832 00000100:00100000:3.0:1789302852.627264:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371160832 00000100:00100000:3.0:1789302852.627279:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1789302852.627285:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371161216:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.627722:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371161088:12345-0@lo:400:kworker.0 Request processed in 1448us (1706us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302852.627732:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1789302852.627749:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371161344 00000100:00100000:2.0:1789302852.627753:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371161344 00000100:00100000:2.0:1789302852.627796:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789302852.627802:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371161344:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.627835:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371161344:12345-0@lo:400:kworker.0 Request processed in 33us (751us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302852.627841:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789302852.627880:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371161088:0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.627944:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371161344:0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.627954:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f047100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371160704:0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.627984:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:2.0:1789302852.627998:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371160832:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302852.628037:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371160832:12345-0@lo:400:kworker.0 Request processed in 46us (991us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302852.628043:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:2.0:1789302852.628069:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371160832:0@lo:400:kworker.0 00000100:00080000:2.0:1789302852.628094:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302852.628096:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302852.629179:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371161216:12345-0@lo:400:kworker.0 Request processed in 1890us (2181us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302852.629188:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1789302852.629267:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371161216:0@lo:400:kworker.0 00000001:00100000:3.0:1789302852.693806:0:5865:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-MDT0000: committing for initial connect of 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000001:00080000:3.0:1789302852.693834:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968186 is committed 00000020:00080000:3.0:1789302852.693843:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 30 batchid updated 00000020:00000080:3.0:1789302852.693858:0:5865:0:(genops.c:885:class_export_put()) final put ffff960f10081800/36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000020:00080000:3.0:1789302852.693879:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30 00000020:00080000:3.0:1789302852.693887:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30 00000020:00000080:3.0:1789302852.693894:0:37:0:(genops.c:844:class_export_destroy()) destroying export ffff960f10081800/36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 for lustre-MDT0000 00000020:00000080:3.0:1789302852.693950:0:37:0:(genops.c:1182:obd_zombie_import_free()) destroying import ffff960f10083000 for lustre-MDT0000 00000001:00100000:2.0:1789302852.724657:0:6719:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000020:00000080:2.0:1789302852.724667:0:6719:0:(genops.c:885:class_export_put()) final put ffff960f10086800/36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000020:00000080:0.0:1789302852.724791:0:8:0:(genops.c:844:class_export_destroy()) destroying export ffff960f10086800/36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 for lustre-OST0000 00000020:00000080:0.0:1789302852.724850:0:8:0:(genops.c:1182:obd_zombie_import_free()) destroying import ffff960f10084800 for lustre-OST0000 00000100:00100000:2.0:1789302853.138601:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371161472:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302853.138701:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302853.139648:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371161472 00000100:00100000:3.0:1789302853.139658:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371161472 00000100:00100000:3.0:1789302853.139723:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:3.0:1789302853.139737:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371161472:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302853.139777:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85000192 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302853.139861:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371161472:12345-0@lo:13:osp-pre-1-0.0 Request processed in 124us (1160us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302853.139868:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:1.0:1789302853.140123:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371161472:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302853.264441:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302853.337431:0:21720:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12r: lseek restores released file ===== 08:34:11 (1789302851) 00000100:00100000:3.0:1789302853.630754:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.630976:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350469120 00000100:00100000:0.0:1789302853.630988:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350469120 00000100:00100000:0.0:1789302853.631071:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1328 00000100:00100000:0.0:1789302853.631083:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:31471:x1876219350469120:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:0.0:1789302853.631181:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:31471:x1876219350469120:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 100us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302853.631189:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1328 00000100:00080000:3.0:1789302853.631487:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302853.645065:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371161600:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302853.645141:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302853.645188:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371161600 00000100:00100000:3.0:1789302853.645208:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371161600 00000100:00100000:3.0:1789302853.645313:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 205 00000100:00100000:3.0:1789302853.645325:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371161600:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302853.645439:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 85000192 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302853.645586:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371161600:12345-0@lo:13:osp-pre-0-0.0 Request processed in 262us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302853.645597:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 205 00000100:00100000:1.0:1789302853.645665:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371161600:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302853.657451:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.658006:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350469248 00000100:00100000:2.0:1789302853.658018:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350469248 00000100:00100000:2.0:1789302853.658110:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1329 00000100:00100000:2.0:1789302853.658120:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:31472:x1876219350469248:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302853.658139:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302853.658156:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef51 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bca expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.658218:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.658233:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.658247:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.658263:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.658270:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.658277:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302853.658716:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302853.659205:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659216:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659224:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659235:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.659239:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659247:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659261:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:2.0:1789302853.659264:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.659270:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659275:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef58 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659292:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659299:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659304:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659309:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.659312:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659322:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659329:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff960f10d3c000) 00010000:00010000:2.0:1789302853.659331:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.659334:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659339:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef5f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659493:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef51 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bca expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659547:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:2.0:1789302853.659550:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.659554:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef51 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bca expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.659562:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2200, rc 0) 00000100:00100000:2.0:1789302853.659640:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:31472:x1876219350469248:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1519us (2198us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302853.659647:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1329 00000100:00100000:3.0:1789302853.663178:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.663389:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350469504 00000100:00100000:2.0:1789302853.663401:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350469504 00000100:00100000:2.0:1789302853.663461:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1330 00000100:00100000:2.0:1789302853.663472:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:31472:x1876219350469504:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:2.0:1789302853.663565:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302853.663593:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f42045a00/0x64fe43095184ec4f lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ac7 expref: 100 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.663613:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:2.0:1789302853.663618:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302853.664047:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.664063:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.664071:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.664104:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 46 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.664120:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 46 pid: 5845 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302853.664135:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371161728:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302853.664402:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b000 going to sleep for 11 seconds 00000100:00100000:2.0:1789302853.667221:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371161728:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302853.667255:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302853.669307:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.669699:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350469632 00000100:00100000:0.0:1789302853.669709:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350469632 00010000:00010000:0.0:1789302853.669762:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 47 pid: 5845 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302853.669803:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 400 00000100:00100000:0.0:1789302853.669813:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:2347:x1876219350469632:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302853.669831:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302853.669838:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 47 pid: 5845 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.669857:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 47 pid: 5845 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.669879:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe43095184ebca lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0a9d expref: 47 pid: 5845 timeout: 790 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.669896:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:0.0:1789302853.669902:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.669911:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.669919:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.669987:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302853.670042:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:2347:x1876219350469632:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 230us (743us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302853.670051:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 400 00010000:00010000:2.0:1789302853.671098:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.671113:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.671140:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.671152:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.671161:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302853.674278:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 16384 00010000:00010000:2.0:1789302853.674297:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.674307:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.674315:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.674321:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.674326:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.674333:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.674625:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94dc00) 00010000:00010000:2.0:1789302853.674628:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.674634:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.674640:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef66 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302853.674663:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dd063f52 reply state 00000000abbbe894 transno 4294968187 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302853.674673:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302853.675057:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:31472:x1876219350469504:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 11581us (11887us total) trans 4294968187 rc 0/0 00000100:00100000:2.0:1789302853.675065:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1330 00010000:00010000:2.0:1789302853.677671:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.677697:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.677704:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.677710:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.677715:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.677723:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.677738:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff960f10d3c000) 00010000:00010000:2.0:1789302853.677742:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.677746:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.677753:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef6d lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302853.692850:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302853.693471:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350469760 00000100:00100000:1.0:1789302853.693483:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350469760 00000100:00100000:1.0:1789302853.693586:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1331 00000100:00100000:1.0:1789302853.693597:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+100:31473:x1876219350469760:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302853.693625:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302853.693651:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184ef74 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bd1 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.693733:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ef7b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.693747:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ef7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.693758:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ef7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302853.693793:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302853.693796:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302853.693813:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ef7b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.693847:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184ef74 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bd1 expref: 102 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.693868:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ef7b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bd1 expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.693875:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302853.693892:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302853.693896:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8400, rc 0) 00000100:00100000:1.0:1789302853.693960:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:31473:x1876219350469760:12345-192.168.203.11@tcp:101:bash.0 Request processed in 365us (1130us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302853.693969:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1331 00000100:00100000:3.0:1789302853.700175:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.701468:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350469888 00000100:00100000:2.0:1789302853.701482:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350469888 00000100:00100000:2.0:1789302853.701868:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1332 00000100:00100000:2.0:1789302853.701891:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+101:31473:x1876219350469888:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302853.701913:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302853.701940:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef82 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bd8 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702003:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702016:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702026:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702040:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702046:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702052:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702144:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef97 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702153:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef97 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702160:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef97 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302853.702191:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 16384 00000004:00200000:2.0:1789302853.702194:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302853.702212:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef97 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702224:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702231:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702238:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702243:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.702248:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702255:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702266:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:2.0:1789302853.702269:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.702274:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702285:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184ef89 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702298:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702304:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702310:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702315:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.702318:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702325:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702331:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff960f10d3c000) 00010000:00010000:2.0:1789302853.702345:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.702349:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702355:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184ef90 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702387:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184ef82 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bd8 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702406:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3a00/0x64fe43095184ef97 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bd8 expref: 102 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.702413:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:2.0:1789302853.702415:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302853.702418:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3a00, rc 0) 00000100:00100000:2.0:1789302853.702490:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:31473:x1876219350469888:12345-192.168.203.11@tcp:101:bash.0 Request processed in 613us (2323us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302853.702497:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1332 00000100:00100000:3.0:1789302853.707143:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.708493:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350470016 00000100:00100000:0.0:1789302853.708528:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350470016 00000100:00100000:0.0:1789302853.708596:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1333 00000100:00100000:0.0:1789302853.708607:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:31473:x1876219350470016:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302853.708629:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302853.708724:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184ef9e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bdf expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.708858:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.708874:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.708885:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.708933:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.708941:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.708948:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709015:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709026:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709033:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709039:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302853.709042:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709049:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709060:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:0.0:1789302853.709063:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302853.709069:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709075:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184efa5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709089:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709095:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709100:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709105:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302853.709108:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709114:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709121:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f09ae1800) 00010000:00010000:0.0:1789302853.709123:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302853.709125:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709131:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184efac lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709167:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184ef9e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bdf expref: 103 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709178:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:0.0:1789302853.709180:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302853.709184:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184ef9e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bdf expref: 103 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.709194:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302853.709265:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:31473:x1876219350470016:12345-192.168.203.11@tcp:101:bash.0 Request processed in 660us (2130us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302853.709272:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1333 00000100:00100000:3.0:1789302853.726350:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.730095:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350470144 00000100:00100000:0.0:1789302853.730113:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350470144 00000100:00100000:1.0:1789302853.734166:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1334 00000100:00100000:1.0:1789302853.734182:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+46:31474:x1876219350470144:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302853.734203:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302853.734227:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184efb3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0be6 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.734298:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.734313:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.734325:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302853.734364:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302853.734367:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302853.734386:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.734412:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184efb3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.734432:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.734442:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302853.734445:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302853.734449:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc 0) 00000100:00100000:1.0:1789302853.734569:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:31474:x1876219350470144:12345-192.168.203.11@tcp:101:rm.0 Request processed in 389us (8228us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302853.734576:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1334 00000100:00100000:3.0:1789302853.741695:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.741793:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350470272 00000100:00100000:2.0:1789302853.741803:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350470272 00000100:00100000:2.0:1789302853.741864:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1335 00000100:00100000:2.0:1789302853.741874:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:31474:x1876219350470272:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302853.741895:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302853.741916:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184efc1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bed expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.741985:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742000:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742011:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742025:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742032:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742039:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742129:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184efd6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742136:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184efd6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742143:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184efd6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302853.742169:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 16384 00000004:00200000:2.0:1789302853.742172:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302853.742189:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184efd6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742200:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742207:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742214:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742220:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.742224:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742232:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742244:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:2.0:1789302853.742246:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.742251:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742257:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184efc8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742270:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742276:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742282:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742287:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.742289:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742295:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742301:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x9699eafa (ffff960f10d3c000) 00010000:00010000:2.0:1789302853.742303:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.742306:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742316:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184efcf lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x9699eafa bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742342:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184efc1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bed expref: 49 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742354:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2a00/0x64fe43095184efd6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bed expref: 48 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.742362:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:2.0:1789302853.742364:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302853.742367:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2a00, rc 0) 00000100:00100000:2.0:1789302853.742423:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31474:x1876219350470272:12345-192.168.203.11@tcp:101:rm.0 Request processed in 552us (756us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302853.742430:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1335 00000100:00100000:3.0:1789302853.752115:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.752454:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350470400 00000100:00100000:2.0:1789302853.752466:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350470400 00000100:00100000:2.0:1789302853.752541:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1336 00000100:00100000:2.0:1789302853.752552:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31474:x1876219350470400:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302853.752575:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302853.752598:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184efdd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bf4 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.752669:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.752927:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.752942:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.752960:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.752967:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.752973:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753028:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753035:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753041:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753046:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.753050:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753057:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753067:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:2.0:1789302853.753070:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.753075:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753082:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184efe4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753096:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753101:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753107:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753112:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.753114:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753120:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753126:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f10d3c900) 00010000:00010000:2.0:1789302853.753129:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.753131:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753137:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184efeb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753172:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184efdd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bf4 expref: 49 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753186:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:2.0:1789302853.753188:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.753191:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184efdd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bf4 expref: 49 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.753200:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302853.753270:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31474:x1876219350470400:12345-192.168.203.11@tcp:101:rm.0 Request processed in 720us (1166us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302853.753277:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1336 00000100:00100000:1.0:1789302853.757005:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302853.757101:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350470528 00000100:00100000:3.0:1789302853.757115:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350470528 00000100:00100000:3.0:1789302853.757174:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1337 00000100:00100000:3.0:1789302853.757183:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31474:x1876219350470528:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302853.757203:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302853.757224:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184eff2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0bfb expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757289:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757303:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757314:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757327:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757333:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757351:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757406:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757413:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757419:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757425:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302853.757429:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757435:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757459:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:3.0:1789302853.757462:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302853.757467:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757472:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184eff9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757486:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757491:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757496:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757500:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302853.757524:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757530:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757540:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302853.757542:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302853.757545:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757550:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f000 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757581:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184eff2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bfb expref: 49 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757590:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:3.0:1789302853.757592:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302853.757594:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184eff2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bfb expref: 49 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.757603:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302853.757660:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31474:x1876219350470528:12345-192.168.203.11@tcp:101:rm.0 Request processed in 478us (665us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302853.757667:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1337 00000040:00080000:1.0:1789302853.774235:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302853.774245:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302853.774284:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 39 count 2 00000040:00080000:1.0:1789302853.774293:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302853.779157:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302853.779345:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350470656 00000100:00100000:1.0:1789302853.779355:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350470656 00000100:00100000:1.0:1789302853.779420:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1338 00000100:00100000:1.0:1789302853.779430:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31475:x1876219350470656:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302853.779452:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302853.779472:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f007 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c02 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779586:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779600:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779613:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779629:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779639:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779645:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779711:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779719:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779726:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779733:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302853.779738:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779745:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779766:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:1.0:1789302853.779769:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302853.779775:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779781:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f00e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779794:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779801:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779806:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779811:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302853.779814:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779820:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779827:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f2c94d200) 00010000:00010000:1.0:1789302853.779830:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302853.779832:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779837:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f015 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779870:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f007 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c02 expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779879:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:1.0:1789302853.779881:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302853.779884:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f007 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c02 expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302853.779897:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8000, rc 0) 00000100:00100000:1.0:1789302853.779958:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31475:x1876219350470656:12345-192.168.203.11@tcp:101:cp.0 Request processed in 530us (810us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302853.779966:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1338 00000100:00100000:1.0:1789302853.783641:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.791914:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350470784 00000100:00100000:2.0:1789302853.791932:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350470784 00000100:00100000:2.0:1789302853.792024:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1339 00000100:00100000:2.0:1789302853.792036:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31475:x1876219350470784:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302853.792057:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302853.792079:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f01c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c09 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792152:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792169:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792180:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792194:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792201:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792208:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792261:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792269:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792276:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792282:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.792286:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792292:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792304:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:2.0:1789302853.792307:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.792311:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792317:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f023 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792330:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792337:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792343:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792348:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302853.792350:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792356:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792361:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f10d3c900) 00010000:00010000:2.0:1789302853.792364:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.792366:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792372:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f02a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792406:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f01c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c09 expref: 49 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792416:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:2.0:1789302853.792418:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302853.792421:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f01c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c09 expref: 49 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.792436:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302853.792527:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31475:x1876219350470784:12345-192.168.203.11@tcp:101:cp.0 Request processed in 493us (8895us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302853.792535:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1339 00000100:00100000:2.0:1789302853.798943:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.799063:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350471040 00000100:00100000:0.0:1789302853.799074:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350471040 00000100:00100000:0.0:1789302853.799138:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1340 00000100:00100000:0.0:1789302853.799148:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31475:x1876219350471040:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302853.799167:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302853.799189:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f031 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c10 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.799321:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.799337:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.799359:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.799376:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.799382:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.799388:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302853.800874:0:18427:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1b:0x0], index 1: 0 00010000:00010000:0.0:1789302853.801244:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801261:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801274:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801312:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801320:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801327:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801736:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302853.801744:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801822:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801838:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:0.0:1789302853.801841:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302853.801846:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801853:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f038 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302853.801867:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000cd66fa44 reply state 00000000ab3a44e0 transno 4294968188 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302853.801876:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789302853.801931:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f031 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 51 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801952:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 50 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.801958:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302853.801961:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302853.801963:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00000100:00100000:0.0:1789302853.802041:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31475:x1876219350471040:12345-192.168.203.11@tcp:101:cp.0 Request processed in 2894us (3106us total) trans 4294968188 rc 0/0 00000100:00100000:0.0:1789302853.802050:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1340 00010000:00010000:3.0:1789302853.806636:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.806655:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.806662:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.806669:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302853.806674:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.806691:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.806707:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f09ae1800) 00010000:00010000:3.0:1789302853.806710:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302853.806715:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.806720:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f03f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302853.814938:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302853.815022:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350471168 00000100:00100000:3.0:1789302853.815032:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350471168 00000100:00100000:3.0:1789302853.815107:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 372 00000100:00100000:3.0:1789302853.815118:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350471168:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302853.815136:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302853.815157:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c17 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302853.815185:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1b:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302853.815204:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=690s v=(0 0) 00010000:00010000:3.0:1789302853.815209:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302853.815218:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.815243:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.815257:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302853.815307:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350471168:12345-192.168.203.11@tcp:101:cp.0 Request processed in 191us (379us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302853.815314:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 372 00000100:00080000:2.0:1789302853.815456:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302853.821059:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302853.821437:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350471296 00000100:00100000:3.0:1789302853.821448:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350471296 00000100:00100000:3.0:1789302853.821529:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 373 00000100:00100000:3.0:1789302853.821541:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:31475:x1876219350471296:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302853.821561:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302853.821579:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c1e expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.821610:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.821620:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302853.821624:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302853.821630:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.821661:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.821675:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 11 pid: 18578 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302853.821698:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371161856:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302853.821792:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58ae40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302853.824017:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef30380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371161856:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302853.824047:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.824066:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:2.0:1789302853.824121:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:31475:x1876219350471296:12345-192.168.203.11@tcp:101:cp.0 Request processed in 2583us (3072us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302853.824130:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 373 00000100:00100000:3.0:1789302853.842098:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.842195:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350471424 00000100:00100000:0.0:1789302853.842205:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350471424 00010000:00010000:0.0:1789302853.842246:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 11 pid: 18578 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302853.842291:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 401 00000100:00100000:0.0:1789302853.842300:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350471424:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302853.842315:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302853.842339:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 11 pid: 18578 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.842363:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 11 pid: 18578 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.842385:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000019b17bd5/0x64fe43095184f054 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c17 expref: 11 pid: 18578 timeout: 790 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.842404:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302853.842408:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302853.842416:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.842426:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.842463:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302853.842477:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371161984:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302853.842698:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302853.851188:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371161984:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302853.851216:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302853.851272:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350471424:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 8972us (9182us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302853.851283:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 401 00000100:00100000:3.0:1789302853.854698:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.854852:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350471680 00000100:00100000:2.0:1789302853.854862:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350471680 00000100:00100000:2.0:1789302853.854923:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 271 00000100:00100000:2.0:1789302853.854932:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31475:x1876219350471680:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302853.855332:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31475:x1876219350471680:12345-192.168.203.11@tcp:35:cp.0 Request processed in 400us (654us total) trans 4294968189 rc 0/0 00000100:00100000:2.0:1789302853.855350:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 271 00000100:00100000:3.0:1789302853.907740:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.907860:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350471808 00000100:00100000:0.0:1789302853.907871:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350471808 00000100:00100000:0.0:1789302853.907948:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1341 00000100:00100000:0.0:1789302853.907959:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:31478:x1876219350471808:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302853.907981:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302853.908003:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f062 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c25 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908070:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908085:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908101:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908116:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908124:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908130:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908226:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908235:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908242:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302853.908275:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00000004:00200000:0.0:1789302853.908279:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302853.908295:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908306:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908314:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908324:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908330:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908335:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302853.908340:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908347:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908359:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:0.0:1789302853.908362:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302853.908368:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908373:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f069 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908388:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908394:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908399:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908404:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302853.908407:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908412:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908419:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f09ae1000) 00010000:00010000:0.0:1789302853.908421:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302853.908424:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908430:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f070 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908459:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f062 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 104 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908473:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 103 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.908479:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302853.908482:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302853.908484:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302853.908611:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:31478:x1876219350471808:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 653us (880us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302853.908619:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1341 00000100:00100000:0.0:1789302853.912825:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.912876:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.912896:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.912943:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302853.913092:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350471936 00000100:00100000:3.0:1789302853.913113:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350471936 00000100:00100000:3.0:1789302853.913210:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:3.0:1789302853.913223:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350471936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302853.913297:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350472064 00000100:00100000:2.0:1789302853.913309:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350472064 00000100:00100000:3.0:1789302853.913358:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350471936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 134us (477us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302853.913369:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:2.0:1789302853.913386:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350472192 00000100:00100000:2.0:1789302853.913392:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350472192 00000100:00100000:3.0:1789302853.913447:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350472320 00000100:00100000:3.0:1789302853.913457:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350472320 00000100:00100000:2.0:1789302853.913459:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 494 00000100:00100000:2.0:1789302853.913485:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350472064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302853.913569:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 495 00000100:00100000:3.0:1789302853.913581:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350472192:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302853.913602:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:2348:x1876219350472064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 126us (783us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302853.913611:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 494 00000100:00100000:2.0:1789302853.913636:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 496 00000100:00100000:2.0:1789302853.913645:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350472320:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302853.913673:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350472320:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (729us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302853.913690:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 496 00000100:00080000:2.0:1789302853.913711:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302853.913715:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302853.913856:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350472192:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 276us (956us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302853.913865:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 495 00000100:00100000:3.0:1789302853.916372:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.918076:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350472448 00000100:00100000:2.0:1789302853.918095:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350472448 00000100:00100000:2.0:1789302853.918185:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 374 00000100:00100000:2.0:1789302853.918197:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350472448:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302853.918226:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302853.918252:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184f07e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c2c expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302853.918480:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371162112:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302853.918906:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b0c0 going to sleep for 11 seconds 00002000:00010000:2.0:1789302853.923302:0:13439:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302853.923318:0:13439:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb mtime: 0 -> 1789302853 00002000:00010000:2.0:1789302853.923321:0:13439:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb atime: 0 -> 1789302853 00002000:00010000:2.0:1789302853.923324:0:13439:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x1b:0x0] updating lvb ctime: 0 -> 1789302853 00000100:00100000:2.0:1789302853.923371:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371162112:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302853.923426:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184f07e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c2c expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.923459:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000d9ee8de/0x64fe43095184f07e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c2c expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302853.923477:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302853.923904:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350472448:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 5707us (7541us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302853.923915:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 374 00000100:00100000:3.0:1789302853.928566:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302853.928944:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350472576 00000100:00100000:0.0:1789302853.928961:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350472576 00000100:00100000:0.0:1789302853.929037:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1342 00000100:00100000:0.0:1789302853.929049:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:31478:x1876219350472576:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302853.929077:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302853.929110:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f085 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c33 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.929205:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.929223:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.929238:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302853.929292:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4d:0x0]: returning size 16384 00000004:00200000:0.0:1789302853.929295:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302853.929315:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.929345:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f085 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 105 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.929366:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 104 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302853.929376:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:0.0:1789302853.929379:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302853.929384:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345bc00, rc 0) 00000100:00100000:0.0:1789302853.929470:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:31478:x1876219350472576:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 422us (918us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302853.929478:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1342 00000100:00100000:1.0:1789302853.933138:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302853.935842:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350472704 00000100:00100000:3.0:1789302853.935862:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350472704 00000100:00100000:3.0:1789302853.936214:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 375 00000100:00100000:3.0:1789302853.936226:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350472704:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302853.936245:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302853.936263:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f093 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c3a expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302853.936330:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371162240:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302853.936440:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58a600 going to sleep for 11 seconds 00000100:00100000:3.0:1789302853.941372:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371162240:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302853.941425:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f093 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c3a expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.941457:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f093 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c3a expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302853.941474:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302853.941607:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350472704:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 5382us (8479us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302853.941618:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 375 00000100:00100000:0.0:1789302853.947542:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302853.947842:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350472960 00000100:00100000:2.0:1789302853.947858:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350472960 00000100:00100000:2.0:1789302853.947981:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1343 00000100:00100000:2.0:1789302853.947995:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:31478:x1876219350472960:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:2.0:1789302853.948403:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].39, 136 off13496 00000100:00100000:2.0:1789302853.948558:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:31478:x1876219350472960:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 566us (1063us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302853.948567:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1343 00010000:00010000:0.0:1789302854.284923:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00100000:1.0:1789302854.763974:0:7654:0:(tgt_lastrcvd.c:563:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of 36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000020:00000080:1.0:1789302854.763982:0:7654:0:(genops.c:885:class_export_put()) final put ffff960f10082800/36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 00000020:00000080:0.0:1789302854.769276:0:8:0:(genops.c:844:class_export_destroy()) destroying export ffff960f10082800/36d456e5-2a1b-4d58-a7f1-78ab3c35acb0 for lustre-OST0001 00000020:00000080:0.0:1789302854.769366:0:8:0:(genops.c:1182:obd_zombie_import_free()) destroying import ffff960f1d83c000 for lustre-OST0001 00000040:00080000:3.0:1789302854.797738:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302854.797746:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302854.797791:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 40 count 3 00000040:00080000:3.0:1789302854.797798:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302854.798597:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371162368:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:3.0:1789302854.798711:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58ab40 going to sleep for 11 seconds 00000100:00100000:3.0:1789302854.808633:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371162368:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302854.809209:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302854.809249:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350473088 00000100:00100000:0.0:1789302854.809256:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350473088 00000100:00100000:0.0:1789302854.809316:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1344 00000100:00100000:0.0:1789302854.809326:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31488:x1876219350473088:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302854.809361:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302854.809475:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31488:x1876219350473088:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 149us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302854.809482:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1344 00000100:00100000:1.0:1789302854.817410:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302854.817603:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350473344 00000100:00100000:3.0:1789302854.817614:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350473344 00000100:00100000:1.0:1789302854.817742:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1345 00000100:00100000:1.0:1789302854.817753:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31488:x1876219350473344:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302854.817777:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302854.817801:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f09a lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c41 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.817903:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f0a1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.817917:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f0a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.817931:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f0a1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302854.817950:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:1.0:1789302854.818072:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f09a lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c41 expref: 54 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.818090:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f0a1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c41 expref: 53 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.818098:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302854.818101:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302854.818104:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9000, rc 0) 00000100:00100000:1.0:1789302854.818168:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31488:x1876219350473344:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 417us (768us total) trans 4294968190 rc 0/0 00000100:00100000:1.0:1789302854.818175:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1345 00000100:00100000:0.0:1789302854.825298:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302854.825414:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.825485:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350473472 00000100:00100000:1.0:1789302854.825496:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350473472 00000100:00100000:1.0:1789302854.825574:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 376 00000100:00100000:1.0:1789302854.825586:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350473472:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302854.825633:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.825654:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.825671:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1b:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302854.825686:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302854.825691:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.825729:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.825744:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 10 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302854.825760:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371162496:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302854.825889:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4d80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302854.827431:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350473600 00000100:00100000:2.0:1789302854.827442:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350473600 00000100:00100000:2.0:1789302854.827559:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 62 00000100:00100000:2.0:1789302854.827569:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2345:x1876219350473600:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302854.827582:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302854.827603:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f0a1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c41 expref: 53 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.827619:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302854.827624:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302854.827662:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350473600:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 95us (2249us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302854.827669:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 62 00000100:00100000:1.0:1789302854.838211:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371162496:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302854.838243:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302854.840735:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.843532:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350473728 00000100:00100000:1.0:1789302854.843547:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350473728 00002000:00100000:1.0:1789302854.843601:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1b:0x0] (0->157) req@ffff960f3f044000 x1876219350473728/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:430/0 lens 648/0 e 0 to 0 dl 1789302865 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302854.843612:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f044000 with x1876219350473728 ext(0->157) 00010000:00010000:1.0:1789302854.843621:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 11 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.843644:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 11 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302854.843652:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3f044000 00000100:00100000:1.0:1789302854.843697:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 12 00000100:00100000:1.0:1789302854.843708:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350473728:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302854.845313:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350473728:12345-192.168.203.11@tcp:4:cp.0 Request processed in 1600us (4582us total) trans 4294967374 rc 0/0 00000100:00100000:1.0:1789302854.845845:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 12 00002000:00100000:1.0:1789302854.845859:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1b:0x0] (0->157) req@ffff960f3f044000 x1876219350473728/t4294967374(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:430/0 lens 648/448 e 0 to 0 dl 1789302865 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302854.845870:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f044000 with x1876219350473728 ext(0->157) 00010000:00010000:1.0:1789302854.845880:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 13 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.845899:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 13 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302854.845907:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3f044000 00000100:00100000:3.0:1789302854.850339:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302854.850403:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350473856 00000100:00100000:0.0:1789302854.850413:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350473856 00010000:00010000:0.0:1789302854.850464:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 13 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302854.850536:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 402 00000100:00100000:0.0:1789302854.850547:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350473856:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302854.850562:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302854.850587:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 13 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302854.850600:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 13 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302854.850623:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184f05b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c1e expref: 13 pid: 8571 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302854.850638:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302854.850642:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302854.850647:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302854.850655:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302854.850696:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302854.850728:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.850742:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302854.850750:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038dad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350473856:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 204us (432us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302854.850765:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 402 00010000:00010000:1.0:1789302854.850788:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.850798:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.850807:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.850815:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302854.850820:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.850830:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.850844:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.850852:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0af lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302854.850916:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350473472:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 25333us (25628us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302854.850929:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 376 00000100:00100000:0.0:1789302854.872000:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.872365:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350474112 00000100:00100000:1.0:1789302854.872376:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350474112 00000100:00100000:1.0:1789302854.872435:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1346 00000100:00100000:1.0:1789302854.872445:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:31488:x1876219350474112:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302854.872597:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:31488:x1876219350474112:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 153us (603us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302854.872604:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1346 00000100:00100000:3.0:1789302854.884068:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.884607:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350474240 00000100:00100000:1.0:1789302854.884618:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350474240 00000100:00100000:1.0:1789302854.884686:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 377 00000100:00100000:1.0:1789302854.884697:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350474240:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302854.884714:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302854.884736:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c48 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302854.884763:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1b:0x0] initial LVB size: 158, mtime: 0x6aa69845, atime: 0x0, ctime: 0x6aa69845, blocks: 0x1 00010000:00010000:1.0:1789302854.884784:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=691s v=(0 0) 00010000:00010000:1.0:1789302854.884789:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302854.884798:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.884813:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.884831:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc 0) 00000100:00100000:1.0:1789302854.884896:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350474240:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 202us (837us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302854.884903:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 377 00000100:00100000:3.0:1789302854.893573:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.894163:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350474496 00000100:00100000:1.0:1789302854.894175:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350474496 00000100:00100000:1.0:1789302854.894240:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1347 00000100:00100000:1.0:1789302854.894251:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:31488:x1876219350474496:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302854.894359:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:31488:x1876219350474496:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 109us (792us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302854.894367:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1347 00000100:00100000:0.0:1789302854.902078:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.902267:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350474624 00000100:00100000:1.0:1789302854.902277:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350474624 00002000:00100000:1.0:1789302854.902329:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1b:0x0] (0->4095) req@ffff960f120ca300 x1876219350474624/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:430/0 lens 488/0 e 0 to 0 dl 1789302865 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302854.902341:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120ca300 with x1876219350474624 ext(0->4095) 00002000:00010000:1.0:1789302854.902347:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f120ca300 00000100:00100000:1.0:1789302854.902381:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 119 00000100:00100000:1.0:1789302854.902390:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350474624:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302854.902652:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120ca300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350474624:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 261us (582us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302854.902660:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 119 00002000:00100000:1.0:1789302854.902667:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1b:0x0] (0->4095) req@ffff960f120ca300 x1876219350474624/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:430/0 lens 488/4536 e 0 to 0 dl 1789302865 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302854.902690:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f120ca300 with x1876219350474624 ext(0->4095) 00002000:00010000:1.0:1789302854.902694:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f120ca300 00000100:00100000:3.0:1789302854.910167:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.910537:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350474752 00000100:00100000:1.0:1789302854.910547:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350474752 00000100:00100000:1.0:1789302854.910611:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1348 00000100:00100000:1.0:1789302854.910621:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:31488:x1876219350474752:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302854.910642:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302854.910662:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0bd lrc: 2/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c4f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.910707:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.910721:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.910733:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.910834:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0bd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 54 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.910854:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.910862:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302854.910866:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302854.910870:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9a00, rc 0) 00000100:00100000:1.0:1789302854.910934:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038da680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31488:x1876219350474752:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 314us (777us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302854.910942:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1348 00000100:00100000:0.0:1789302854.935537:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302854.936385:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350474880 00000100:00100000:2.0:1789302854.936396:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350474880 00000100:00100000:2.0:1789302854.936451:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1349 00000100:00100000:2.0:1789302854.936461:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31488:x1876219350474880:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302854.936492:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302854.936615:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31488:x1876219350474880:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 155us (1113us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302854.936624:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1349 00000040:00080000:1.0:1789302854.961276:0:21771:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302854.961286:0:21771:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:0.0:1789302854.961861:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302854.961956:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350475008 00000100:00100000:2.0:1789302854.961967:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350475008 00000100:00100000:2.0:1789302854.962033:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 378 00000100:00100000:2.0:1789302854.962044:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350475008:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302854.962088:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962104:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302854.962110:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302854.962116:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962130:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962160:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962169:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962178:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962186:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302854.962190:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962199:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962213:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.962221:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f0cb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302854.962307:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350475008:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 265us (450us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302854.962315:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 378 00000040:00080000:1.0:1789302854.962447:0:21771:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 40 count 3 00000040:00080000:1.0:1789302854.962454:0:21771:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302854.978957:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302854.979048:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350475264 00000100:00100000:1.0:1789302854.979060:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350475264 00000100:00100000:1.0:1789302854.979127:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1350 00000100:00100000:1.0:1789302854.979138:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31488:x1876219350475264:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1789302854.979221:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.979240:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.979250:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.979286:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302854.979300:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 53 pid: 5845 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302854.979314:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371162624:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302854.979402:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302854.993335:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302854.994083:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371162624:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302854.994117:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302854.994314:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350475392 00000100:00100000:2.0:1789302854.994322:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350475392 00010000:00010000:2.0:1789302854.994360:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 54 pid: 5845 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302854.994404:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 403 00000100:00100000:2.0:1789302854.994413:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2345:x1876219350475392:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302854.994428:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302854.994434:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 1s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 54 pid: 5845 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.994446:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 54 pid: 5845 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.994468:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f0c4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c4f expref: 54 pid: 5845 timeout: 791 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.994480:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302854.994485:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.994495:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.994530:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302854.994555:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302854.994609:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2345:x1876219350475392:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 196us (1284us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302854.994617:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 403 00010000:00010000:3.0:1789302854.994635:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.994643:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.997246:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.997261:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.997268:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.997274:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302854.997278:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.997287:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.997354:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302854.997358:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302854.997362:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302854.997369:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f0d2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:2.0:1789302854.999229:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302854.999236:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302854.999777:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:2.0:1789302854.999786:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302854.999910:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31488:x1876219350475264:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 19154us (19344us total) trans 4294968191 rc 0/0 00000100:00100000:1.0:1789302854.999925:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1350 00000100:00100000:3.0:1789302855.005628:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302855.005726:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350475648 00000100:00100000:1.0:1789302855.005736:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350475648 00000100:00100000:1.0:1789302855.005811:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 272 00000100:00100000:1.0:1789302855.005822:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e9050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31488:x1876219350475648:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302855.006067:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e9050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:31488:x1876219350475648:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 245us (446us total) trans 4294968192 rc 0/0 00000100:00100000:1.0:1789302855.006075:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 272 00000100:00080000:3.0:1789302855.006375:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302855.308970:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302856.332706:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302856.854086:0:21823:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302856.854096:0:21823:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302856.854141:0:21823:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 40 count 3 00000040:00080000:0.0:1789302856.854148:0:21823:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302856.922656:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302856.922772:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350475776 00000100:00100000:0.0:1789302856.922784:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350475776 00000100:00100000:0.0:1789302856.922857:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 379 00000100:00100000:0.0:1789302856.922868:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350475776:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302856.922886:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302856.922904:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c56 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.922931:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=2s v=(0 0) 00010000:00010000:0.0:1789302856.922935:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302856.922943:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.922957:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.922970:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ac00, rc 0) 00000100:00100000:0.0:1789302856.923020:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2345:x1876219350475776:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 155us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302856.923027:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 379 00000100:00080000:3.0:1789302856.923131:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302856.935824:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302856.936778:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350476032 00000100:00100000:1.0:1789302856.936790:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350476032 00000100:00100000:1.0:1789302856.936858:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1351 00000100:00100000:1.0:1789302856.936869:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:31498:x1876219350476032:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302856.936888:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302856.936908:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f0e0 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c5d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.937005:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f0e7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.937020:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f0e7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.937030:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f0e7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302856.937046:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:1.0:1789302856.937167:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f0e0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c5d expref: 107 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.937184:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f0e7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c5d expref: 106 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.937193:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302856.937195:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:1.0:1789302856.937198:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9400, rc 0) 00000100:00100000:1.0:1789302856.937257:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:31498:x1876219350476032:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 390us (1442us total) trans 4294968193 rc 0/0 00000100:00100000:1.0:1789302856.937264:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1351 00000100:00080000:1.0:1789302856.937286:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302856.946384:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302856.948497:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350476160 00000100:00100000:1.0:1789302856.948532:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350476160 00000100:00100000:0.0:1789302856.948654:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 380 00000100:00100000:0.0:1789302856.948678:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2345:x1876219350476160:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:0.0:1789302856.948725:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.948752:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.948761:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.948769:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302856.948774:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302856.948779:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.948816:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.948831:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 18 pid: 6695 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302856.948844:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371162752:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302856.948946:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.948963:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 11 pid: 6697 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302856.948973:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371162880:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302856.948998:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f02cd9900 going to sleep for 11 seconds 00000100:00100000:0.0:1789302856.951164:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371162752:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302856.951178:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371162880:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302856.951205:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302856.951762:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302856.951793:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302856.951911:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350476288 00000100:00100000:1.0:1789302856.951918:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350476288 00010000:00010000:1.0:1789302856.951954:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 19 pid: 6695 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302856.951979:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:1.0:1789302856.951988:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2348:x1876219350476288:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302856.952002:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302856.952024:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 19 pid: 6695 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952042:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 19 pid: 6695 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952062:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184f0d9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c56 expref: 19 pid: 6695 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952079:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302856.952124:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350476288:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 137us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302856.952132:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:1.0:1789302856.952148:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350476416 00000100:00100000:1.0:1789302856.952150:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350476416 00010000:00010000:1.0:1789302856.952171:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 12 pid: 6697 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302856.952195:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 405 00000100:00100000:1.0:1789302856.952201:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120bb050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350476416:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302856.952208:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302856.952216:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 12 pid: 6697 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952226:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 12 pid: 6697 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952245:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095184f0b6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c48 expref: 12 pid: 6697 timeout: 793 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952255:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302856.952260:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302856.952265:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952274:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952298:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302856.952318:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120bb050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350476416:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 116us (524us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302856.952324:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 405 00000100:00080000:1.0:1789302856.952372:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1789302856.952384:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952394:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952418:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952432:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952440:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952448:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302856.952452:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952461:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952472:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.952480:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095184f0f5 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302856.952539:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350476160:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 3865us (6165us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302856.952547:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 380 00000100:00100000:1.0:1789302856.974246:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302856.974326:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350476672 00000100:00100000:3.0:1789302856.974335:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350476672 00000100:00100000:3.0:1789302856.974389:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 273 00000100:00100000:3.0:1789302856.974400:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:31498:x1876219350476672:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:3.0:1789302856.974461:0:12976:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x28:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f0e7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c5d expref: 106 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302856.974722:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968194 00000001:00080000:3.0:1789302856.975232:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968195 00010000:00010000:3.0:1789302856.975290:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302856.975307:0:12976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302856.975322:0:12976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 109 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302856.975331:0:12976:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 5/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302856.975367:0:12976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 109 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302856.975382:0:12976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 109 pid: 6919 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302856.975396:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371163008:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302856.975493:0:12976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302856.975523:0:12976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 53 pid: 18427 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302856.975534:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266ed80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371163136:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302856.975562:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58a6c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302856.979032:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302856.979068:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302856.980870:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350476800 00000100:00100000:2.0:1789302856.980883:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350476800 00010000:00010000:2.0:1789302856.980927:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 110 pid: 6919 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302856.980965:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350476928 00000100:00100000:2.0:1789302856.980969:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350476928 00010000:00010000:2.0:1789302856.980984:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 54 pid: 18427 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302856.981007:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 406 00000100:00100000:2.0:1789302856.981017:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2348:x1876219350476800:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302856.981032:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302856.981038:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 110 pid: 6919 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981056:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 110 pid: 6919 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981076:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302856.981082:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981090:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981097:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302856.981155:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2348:x1876219350476800:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 140us (2131us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302856.981162:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 406 00000100:00100000:2.0:1789302856.981178:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 407 00000100:00100000:2.0:1789302856.981184:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2347:x1876219350476928:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302856.981192:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302856.981196:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 54 pid: 18427 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981205:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 54 pid: 18427 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981216:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302856.981219:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981232:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981240:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302856.981260:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302856.981279:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2347:x1876219350476928:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 94us (2210us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302856.981285:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 407 00010000:00010000:1.0:1789302856.986875:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f077 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c25 expref: 109 pid: 6919 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302856.986908:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038dad80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371163008:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302856.986923:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f046 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c10 expref: 53 pid: 18427 timeout: 793 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302856.986934:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266ed80 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371163136:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302856.986960:0:12976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.986970:0:12976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302856.986976:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:1.0:1789302856.987107:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968196 00000001:00080000:1.0:1789302856.987311:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968197 00000001:00080000:1.0:1789302856.987388:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968198 00000100:00100000:1.0:1789302856.987472:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371163264:0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:1.0:1789302856.987607:0:12976:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302856.987642:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1d80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302856.988293:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371163264 00000100:00100000:2.0:1789302856.988303:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371163264 00000100:00100000:2.0:1789302856.988393:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 65 00000100:00100000:2.0:1789302856.988402:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371163264:12345-0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:2.0:1789302856.988485:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371163264:12345-0@lo:1000:mdt_rdpg00_002.0 Request processed in 84us (878us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302856.988491:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 65 00000100:00080000:1.0:1789302856.989449:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302856.989493:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371163264:0@lo:1000:mdt_rdpg00_002.0 00000001:00080000:1.0:1789302856.990092:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968199 00000100:00100000:1.0:1789302856.990179:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371163392:0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:1.0:1789302856.990232:0:12976:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302856.990302:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f103d1780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302856.997331:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371163392 00000100:00100000:2.0:1789302856.997346:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371163392 00000100:00100000:2.0:1789302856.997406:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:2.0:1789302856.997414:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371163392:12345-0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:2.0:1789302856.997790:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:12976:x1876219371163392:12345-0@lo:1000:mdt_rdpg00_002.0 Request processed in 375us (7557us total) trans 4294967375 rc 0/0 00000100:00100000:2.0:1789302856.997799:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00000100:00100000:0.0:1789302856.997929:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371163392:0@lo:1000:mdt_rdpg00_002.0 00010000:00010000:0.0:1789302856.997960:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.997971:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.997979:0:12976:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.997986:0:12976:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302856.997990:0:12976:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.998000:0:12976:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.998016:0:12976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302856.998020:0:12976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:0.0:1789302856.998025:0:12976:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 2/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302856.998033:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f0fc lrc: 0/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302856.998252:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x4e:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:0.0:1789302856.998366:0:12976:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].13, 64 off9096 00000004:00080000:0.0:1789302856.998379:0:12976:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 694008876271ns, next at 695008874157ns, rc = 0 00000001:00080000:0.0:1789302856.998387:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968200 00010000:00010000:0.0:1789302856.998432:0:12976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302856.998436:0:12976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302856.998443:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f0e7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c5d expref: 113 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302856.998558:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:31498:x1876219350476672:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 24159us (24320us total) trans 4294968201 rc 0/0 00000100:00100000:0.0:1789302856.998567:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 273 00000020:00080000:0.0:1789302856.998571:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:1.0:1789302857.054135:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302857.054180:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350477184 00000100:00100000:1.0:1789302857.054190:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350477184 00000100:00100000:1.0:1789302857.054243:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1352 00000100:00100000:1.0:1789302857.054252:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:31499:x1876219350477184:12345-192.168.203.11@tcp:101:lseek_test.0 00010000:00010000:1.0:1789302857.054276:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302857.054296:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f103 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c64 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054371:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054386:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054398:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054413:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054421:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054428:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054573:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f118 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054582:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f118 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054590:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f118 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302857.054599:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:1.0:1789302857.054709:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054716:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054722:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054727:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302857.054731:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054739:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054750:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:1.0:1789302857.054752:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302857.054757:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054762:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f10a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054775:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054784:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054789:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054794:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302857.054797:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054803:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054809:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x5fded3f1 (ffff960f2c94d800) 00010000:00010000:1.0:1789302857.054811:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302857.054814:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054819:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f111 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x5fded3f1 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054880:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f103 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c64 expref: 115 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054895:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f118 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c64 expref: 114 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.054903:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f2c94d200) 00010000:00010000:1.0:1789302857.054905:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302857.054908:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9600, rc 0) 00000100:00100000:1.0:1789302857.054962:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:31499:x1876219350477184:12345-192.168.203.11@tcp:101:lseek_test.0 Request processed in 710us (836us total) trans 4294968202 rc 0/0 00000100:00100000:1.0:1789302857.054970:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1352 00000100:00100000:1.0:1789302857.065782:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302857.065824:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350477440 00000100:00100000:1.0:1789302857.065833:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350477440 00000100:00100000:1.0:1789302857.065888:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 63 00000100:00100000:1.0:1789302857.065898:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:31499:x1876219350477440:12345-192.168.203.11@tcp:103:lseek_test.0 00010000:00010000:1.0:1789302857.065914:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302857.065938:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f118 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c64 expref: 114 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302857.065955:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f2c94d200) 00010000:00010000:1.0:1789302857.065966:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302857.066012:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:31499:x1876219350477440:12345-192.168.203.11@tcp:103:lseek_test.0 Request processed in 115us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302857.066020:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 63 00000100:00100000:1.0:1789302857.073615:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302857.074100:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350477568 00000100:00100000:3.0:1789302857.074109:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350477568 00000100:00100000:3.0:1789302857.074161:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1353 00000100:00100000:3.0:1789302857.074171:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:31499:x1876219350477568:12345-192.168.203.11@tcp:58:lseek_test.0 00010000:00010000:3.0:1789302857.074270:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302857.074285:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302857.074297:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302857.074455:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].40, 136 off13632 00000100:00100000:3.0:1789302857.074556:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:31499:x1876219350477568:12345-192.168.203.11@tcp:58:lseek_test.0 Request processed in 386us (947us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302857.074565:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1353 00000100:00100000:1.0:1789302857.079606:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302857.079777:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350477696 00000100:00100000:2.0:1789302857.079797:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350477696 00000100:00100000:2.0:1789302857.079863:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1354 00000100:00100000:2.0:1789302857.079874:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:31499:x1876219350477696:12345-192.168.203.11@tcp:101:lseek_test.0 00010000:00010000:2.0:1789302857.079893:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302857.079914:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c6b expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302857.079970:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302857.079981:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 4/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c6b expref: 114 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302857.080000:0:6919:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302857.080015:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 3/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c6b expref: 114 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302857.080029:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:2.0:1789302857.080033:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302857.080038:0:6919:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 3/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c6b expref: 114 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302857.080046:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2200, rc 0) 00000100:00100000:2.0:1789302857.080096:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:31499:x1876219350477696:12345-192.168.203.11@tcp:101:lseek_test.0 Request processed in 224us (499us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302857.080103:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1354 00010000:00010000:0.0:1789302857.361050:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302857.742887:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371163648:0@lo:400:kworker.0 00000100:00100000:3.0:1789302857.742956:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302857.743021:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1789302857.743027:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371163520:0@lo:400:kworker.0 00000100:00100000:3.0:1789302857.743057:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302857.743091:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371164032:0@lo:400:kworker.0 00000100:00100000:3.0:1789302857.743114:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302857.743140:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371163776:0@lo:400:kworker.0 00000100:00100000:3.0:1789302857.743163:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302857.743180:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302857.743186:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371163904:0@lo:400:kworker.0 00000100:00100000:3.0:1789302857.743207:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302857.743232:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371163648 00000100:00100000:3.0:1789302857.743239:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371163648 00000100:00100000:3.0:1789302857.743275:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371164032 00000100:00100000:0.0:1789302857.743276:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371163776 00000100:00100000:3.0:1789302857.743278:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371164032 00000100:00100000:0.0:1789302857.743280:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371163776 00000100:00100000:3.0:1789302857.743302:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:0.0:1789302857.743307:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:3.0:1789302857.743310:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371163648:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302857.743312:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371164032:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302857.743356:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371164032:12345-0@lo:400:kworker.0 Request processed in 43us (241us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302857.743363:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00100000:0.0:1789302857.743377:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1789302857.743384:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371163776:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789302857.743402:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302857.743413:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371163776:12345-0@lo:400:kworker.0 Request processed in 31us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302857.743419:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:2.0:1789302857.743489:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371163520 00000100:00100000:2.0:1789302857.743493:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371163520 00000100:00100000:2.0:1789302857.743537:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:2.0:1789302857.743544:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371163520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302857.743946:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371163776:0@lo:400:kworker.0 00000100:00100000:1.0:1789302857.744018:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371163904 00000100:00100000:1.0:1789302857.744022:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371163904 00000100:00100000:1.0:1789302857.744057:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:1.0:1789302857.744063:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371163904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302857.744104:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371163904:12345-0@lo:400:kworker.0 Request processed in 41us (896us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302857.744110:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00080000:1.0:1789302857.744124:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302857.744346:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371163904:0@lo:400:kworker.0 00000100:00100000:3.0:1789302857.744670:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371163648:12345-0@lo:400:kworker.0 Request processed in 1359us (1712us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302857.744678:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:0.0:1789302857.748803:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de6300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371164032:0@lo:400:kworker.0 00000100:00100000:0.0:1789302857.749235:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de7480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371163648:0@lo:400:kworker.0 00000100:00100000:2.0:1789302857.756747:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371163520:12345-0@lo:400:kworker.0 Request processed in 13193us (13678us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302857.756762:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:0.0:1789302857.759967:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371163520:0@lo:400:kworker.0 00000001:00080000:2.0:1789302857.812121:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968201 is committed 00000004:00080000:2.0:1789302857.812135:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 15 committed 00000001:00080000:2.0:1789302857.812156:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968188 is committed 00000001:00080000:2.0:1789302857.812164:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968202 is committed 00000020:00080000:2.0:1789302857.812176:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f103d1d80 refcount 2 committed 0 result 0 batchid 31 00000020:00080000:2.0:1789302857.812181:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3fc9db00 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f103d1240 00000020:00080000:2.0:1789302857.812184:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3fc9d000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f109f3000 00000001:00080000:2.0:1789302857.812211:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968192 is committed 00000020:00080000:2.0:1789302857.812246:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 31 committed batchid 30 00000020:00080000:2.0:1789302857.812429:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 30 00000100:00100000:2.0:1789302857.813364:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371164160:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789302857.813434:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302857.813565:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371164160 00000100:00100000:0.0:1789302857.813573:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371164160 00000100:00100000:0.0:1789302857.813627:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 381 00000100:00100000:0.0:1789302857.813636:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371164160:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1789302857.813673:0:6695:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:27 count 1 00002000:00010000:0.0:1789302857.813714:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1b:0x0] initial LVB size: 158, mtime: 0x6aa69845, atime: 0x0, ctime: 0x6aa69845, blocks: 0x1 00010000:00010000:0.0:1789302857.813730:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302857.813745:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=694s v=(0 0) 00010000:00010000:0.0:1789302857.813749:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1b:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302857.813755:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302857.813774:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302857.813783:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302857.813791:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302857.813808:0:6695:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1789302857.814272:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371164160:12345-0@lo:6:osp-syn-1-0.0 Request processed in 635us (835us total) trans 4294967376 rc 0/0 00000100:00100000:0.0:1789302857.814280:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 381 00010000:00010000:3.0:1789302857.814525:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302857.814533:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302857.814543:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302857.814547:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302857.814557:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302857.814574:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302857.814581:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe43095184f134 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302857.814602:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302857.814603:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302857.814605:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1789302857.815047:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f11de5880/2, rc 0, transno 4294967376 00000100:00100000:2.0:1789302857.815056:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371164160:0@lo:6:osp-syn-1-0.0 00000040:00080000:1.0:1789302858.061077:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302858.061087:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302858.061132:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 41 count 4 00000040:00080000:1.0:1789302858.061140:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302858.061262:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320e00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371164288:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302858.062862:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f02cd9900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302858.072603:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320e00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371164288:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302858.075704:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.076436:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350477824 00000100:00100000:0.0:1789302858.076447:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350477824 00000100:00100000:0.0:1789302858.076559:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1355 00000100:00100000:0.0:1789302858.076570:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31500:x1876219350477824:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302858.076603:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302858.076742:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31500:x1876219350477824:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 173us (1053us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.076750:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1355 00000100:00080000:1.0:1789302858.078573:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302858.085170:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302858.085304:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350477952 00000100:00100000:3.0:1789302858.085322:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350477952 00000100:00100000:3.0:1789302858.085378:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1356 00000100:00100000:3.0:1789302858.085389:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31500:x1876219350477952:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:3.0:1789302858.085470:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085488:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085543:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085560:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085568:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085574:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085598:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085606:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085613:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302858.085662:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00000004:00200000:3.0:1789302858.085668:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302858.085680:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085691:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085698:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085705:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085711:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302858.085716:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085891:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085904:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:3.0:1789302858.085907:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302858.085913:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085919:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f13b lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085933:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085940:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085950:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085955:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302858.085957:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085962:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085969:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0xefe2aa85 (ffff960f0cdf2c00) 00010000:00010000:3.0:1789302858.085971:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302858.085974:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085980:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f142 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0xefe2aa85 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085993:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.085999:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.086004:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.086009:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302858.086012:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.086022:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.086028:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:3.0:1789302858.086031:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302858.086033:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.086039:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f149 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.086110:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038dad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31500:x1876219350477952:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 722us (948us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302858.086118:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1356 00000100:00100000:3.0:1789302858.089921:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.090160:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350478208 00000100:00100000:0.0:1789302858.090170:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350478208 00000100:00100000:0.0:1789302858.090231:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1357 00000100:00100000:0.0:1789302858.090241:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31500:x1876219350478208:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302858.090264:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302858.090285:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f150 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c72 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090384:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090400:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090411:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302858.090431:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x28:0x0]: returning size 158 00010000:00010000:0.0:1789302858.090595:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090602:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090609:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090614:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302858.090618:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090624:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090635:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:0.0:1789302858.090638:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302858.090643:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.090648:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f157 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x28:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.092410:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f150 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c72 expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.092884:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302858.092888:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302858.092894:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f150 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c72 expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.092914:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a800, rc 0) 00000100:00100000:0.0:1789302858.093037:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:31500:x1876219350478208:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2796us (3124us total) trans 4294968203 rc 301/301 00000100:00100000:0.0:1789302858.093047:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1357 00000100:00100000:1.0:1789302858.103475:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.104071:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350478336 00000100:00100000:2.0:1789302858.104081:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350478336 00000100:00100000:2.0:1789302858.104174:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1358 00000100:00100000:2.0:1789302858.104185:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:31500:x1876219350478336:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302858.104224:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x28:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302858.104347:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:31500:x1876219350478336:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 163us (881us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.104354:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1358 00000100:00100000:1.0:1789302858.108861:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302858.113024:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350478592 00000100:00100000:3.0:1789302858.113040:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350478592 00000100:00100000:2.0:1789302858.113320:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1359 00000100:00100000:2.0:1789302858.113331:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:31500:x1876219350478592:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302858.113353:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302858.113375:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f165 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c79 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.114243:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f16c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.114259:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f16c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.114271:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f16c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302858.114281:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x29:0x0]: returning size 0 00010000:00010000:2.0:1789302858.114332:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f165 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c79 expref: 52 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.114351:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f16c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c79 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.114361:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c900) 00010000:00010000:2.0:1789302858.114364:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302858.114367:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:2.0:1789302858.114442:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31500:x1876219350478592:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1113us (5590us total) trans 4294968204 rc 0/0 00000100:00100000:2.0:1789302858.114451:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1359 00000100:00100000:1.0:1789302858.127025:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.127158:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350478848 00000100:00100000:0.0:1789302858.127168:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350478848 00000100:00100000:0.0:1789302858.127224:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1360 00000100:00100000:0.0:1789302858.127235:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31500:x1876219350478848:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302858.127291:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302858.127329:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f16c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c79 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.127349:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c900) 00010000:00010000:0.0:1789302858.127362:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302858.127384:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.127398:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.127407:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 4/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.127439:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.127452:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 102 pid: 5843 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302858.127467:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371164416:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302858.127590:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdb3c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302858.129676:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371164416:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302858.129703:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302858.134566:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.134949:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350478976 00000100:00100000:2.0:1789302858.134959:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350478976 00010000:00010000:2.0:1789302858.135019:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 103 pid: 5843 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302858.135063:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 408 00000100:00100000:2.0:1789302858.135073:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2347:x1876219350478976:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302858.135087:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302858.135095:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 103 pid: 5843 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.135107:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 103 pid: 5843 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.135129:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f08c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4d:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c33 expref: 103 pid: 5843 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.135148:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:2.0:1789302858.135153:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.135163:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.135171:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.135209:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302858.135268:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:2347:x1876219350478976:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 196us (712us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.135275:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 408 00010000:00010000:0.0:1789302858.135678:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.135689:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.135767:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.135778:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.135787:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.135993:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136002:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136009:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136015:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302858.136020:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136027:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136037:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:0.0:1789302858.136040:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302858.136044:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136050:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f17a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136061:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136068:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136075:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136081:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302858.136084:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136090:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136097:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0xf99b4243 (ffff960f09ae1f00) 00010000:00010000:0.0:1789302858.136099:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302858.136102:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.136108:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f181 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4d:0x0].0xf99b4243 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302858.136185:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31500:x1876219350478848:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 8950us (9167us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302858.136193:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1360 00000100:00080000:1.0:1789302858.137400:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302858.140331:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.140423:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350479104 00000100:00100000:2.0:1789302858.140435:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350479104 00000100:00100000:2.0:1789302858.140494:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1361 00000100:00100000:2.0:1789302858.140537:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31500:x1876219350479104:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302858.140557:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302858.140580:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f188 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c80 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.140614:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.140628:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.140639:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.140753:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f188 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 52 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.140771:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.140779:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:2.0:1789302858.140783:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302858.140787:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302858.140848:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31500:x1876219350479104:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 345us (526us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.140856:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1361 00000100:00100000:3.0:1789302858.149995:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.150082:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350479360 00000100:00100000:2.0:1789302858.150093:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350479360 00000100:00100000:2.0:1789302858.150168:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1362 00000100:00100000:2.0:1789302858.150179:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31500:x1876219350479360:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302858.150269:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.150288:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.150302:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302858.150563:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e1a2081b reply state 00000000bb001dea transno 4294968205 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302858.150573:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302858.150675:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31500:x1876219350479360:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 497us (690us total) trans 4294968205 rc 0/0 00000100:00100000:2.0:1789302858.150681:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1362 00010000:00010000:2.0:1789302858.153792:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.153809:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.153822:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.153828:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302858.153833:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.153840:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.153856:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:2.0:1789302858.153860:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789302858.153864:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.153871:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f196 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.154029:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.154103:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350479488 00000100:00100000:0.0:1789302858.154114:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350479488 00000100:00100000:0.0:1789302858.154180:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1363 00000100:00100000:0.0:1789302858.154192:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:31500:x1876219350479488:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302858.154217:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302858.154238:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c87 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.154297:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.154310:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.154318:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302858.154321:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302858.154323:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b400, rc 0) 00000100:00100000:0.0:1789302858.154385:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31500:x1876219350479488:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 196us (368us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.154392:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1363 00000100:00100000:3.0:1789302858.156597:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.156716:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350479616 00000100:00100000:1.0:1789302858.156731:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350479616 00000100:00100000:1.0:1789302858.156807:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 274 00000100:00100000:1.0:1789302858.156817:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31500:x1876219350479616:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:1.0:1789302858.156958:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31500:x1876219350479616:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 142us (373us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302858.156965:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 274 00000100:00100000:3.0:1789302858.166697:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.166809:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350479872 00000100:00100000:0.0:1789302858.166824:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350479872 00000100:00100000:0.0:1789302858.166903:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 275 00000100:00100000:0.0:1789302858.166914:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:31500:x1876219350479872:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302858.167605:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:31500:x1876219350479872:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 690us (921us total) trans 4294968206 rc 0/0 00000100:00100000:0.0:1789302858.167614:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 275 00000100:00100000:2.0:1789302858.177696:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.178316:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350480128 00000100:00100000:1.0:1789302858.178328:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350480128 00000100:00100000:1.0:1789302858.178424:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1364 00000100:00100000:1.0:1789302858.178435:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:31500:x1876219350480128:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302858.178582:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:31500:x1876219350480128:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 148us (895us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302858.178590:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1364 00000100:00100000:0.0:1789302858.182709:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.182894:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350480384 00000100:00100000:1.0:1789302858.182903:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350480384 00000100:00100000:1.0:1789302858.183008:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1365 00000100:00100000:1.0:1789302858.183020:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:31500:x1876219350480384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302858.183040:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302858.183059:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f1a4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c8e expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.183458:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.183475:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.183494:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 4/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.183574:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.183590:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 55 pid: 5845 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302858.183612:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbe300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371164544:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302858.183705:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f02cd9900 going to sleep for 11 seconds 00000100:00100000:3.0:1789302858.195356:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.195453:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350480512 00000100:00100000:0.0:1789302858.195472:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350480512 00000100:00100000:1.0:1789302858.195558:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbe300 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371164544:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302858.195583:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 56 pid: 5845 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.195591:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302858.195669:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 409 00000100:00100000:0.0:1789302858.195681:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2348:x1876219350480512:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302858.195701:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302858.195708:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 56 pid: 5845 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.195722:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 56 pid: 5845 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.195749:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f19d lrc: 0/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c87 expref: 56 pid: 5845 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.196066:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302858.196074:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.196088:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.196097:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.196137:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302858.196270:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2348:x1876219350480512:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 590us (921us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.196280:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 409 00010000:00010000:1.0:1789302858.197049:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.197062:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302858.197089:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x29:0x0]: returning size 0 00000004:00080000:1.0:1789302858.197536:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1c:0x0], index 0: 0 00010000:00010000:1.0:1789302858.197719:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.197729:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.197738:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.197745:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302858.197750:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.197759:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.197778:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:1.0:1789302858.197780:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302858.197786:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 2/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.197792:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1ab lrc: 0/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.199071:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f1a4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c8e expref: 55 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.199093:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4d:0x0].0x0 (ffff960f10d3c100) 00010000:00010000:1.0:1789302858.199097:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302858.199103:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f1a4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4d:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c8e expref: 55 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.199117:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8e00, rc 0) 00000100:00100000:1.0:1789302858.199197:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:31500:x1876219350480384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 16180us (16499us total) trans 4294968207 rc 301/301 00000100:00100000:1.0:1789302858.199207:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1365 00000100:00100000:2.0:1789302858.206734:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302858.210607:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350480768 00000100:00100000:3.0:1789302858.210630:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350480768 00000100:00100000:3.0:1789302858.210731:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 276 00000100:00100000:3.0:1789302858.210745:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:31500:x1876219350480768:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302858.211404:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:31500:x1876219350480768:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 659us (4683us total) trans 4294968208 rc 0/0 00000100:00100000:3.0:1789302858.211418:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 276 00000100:00100000:2.0:1789302858.218637:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.218950:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350480896 00000100:00100000:1.0:1789302858.218964:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350480896 00000100:00100000:1.0:1789302858.219044:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1366 00000100:00100000:1.0:1789302858.219057:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:31500:x1876219350480896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302858.219093:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302858.219123:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1b9 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c95 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.219230:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1c0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.219249:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1c0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.219263:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1c0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302858.219311:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x29:0x0]: returning size 0 00000004:00200000:1.0:1789302858.219316:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302858.219340:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1c0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.219372:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f1b9 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c95 expref: 57 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.219394:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1c0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c95 expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.219405:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:1.0:1789302858.219409:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:1.0:1789302858.219413:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8200, rc 0) 00000100:00100000:1.0:1789302858.219547:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:31500:x1876219350480896:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 450us (881us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302858.219556:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1366 00000100:00100000:1.0:1789302858.225635:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.226018:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350481024 00000100:00100000:0.0:1789302858.226032:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350481024 00000100:00100000:0.0:1789302858.226167:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 382 00000100:00100000:0.0:1789302858.226180:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350481024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302858.226205:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302858.226233:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0c9c expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302858.226271:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302858.226299:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=695s v=(0 0) 00010000:00010000:0.0:1789302858.226315:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302858.226328:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.226351:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.226367:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b400, rc 0) 00000100:00100000:0.0:1789302858.226454:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350481024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 276us (833us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.226462:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 382 00000100:00080000:1.0:1789302858.229530:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302858.231299:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.231413:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350481280 00000100:00100000:1.0:1789302858.231429:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350481280 00000100:00100000:1.0:1789302858.231601:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1367 00000100:00100000:1.0:1789302858.231614:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:31500:x1876219350481280:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302858.231796:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:31500:x1876219350481280:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 182us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302858.232465:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1367 00000100:00100000:3.0:1789302858.238963:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.239313:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350481408 00000100:00100000:1.0:1789302858.239328:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350481408 00000100:00100000:1.0:1789302858.239404:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00100000:1.0:1789302858.239416:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:31500:x1876219350481408:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302858.239444:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302858.239469:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ca3 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.239540:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.239550:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302858.239555:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302858.239560:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.239602:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.239619:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 11 pid: 6697 timeout: 795 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302858.239636:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371164672:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302858.241285:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e58a3c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302858.247292:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.248011:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371164672:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302858.248057:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.248084:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8800, rc 0) 00000100:00100000:2.0:1789302858.248183:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:31500:x1876219350481408:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 8768us (9233us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.248197:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 383 00000100:00100000:3.0:1789302858.250122:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350481536 00000100:00100000:3.0:1789302858.250143:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350481536 00010000:00010000:3.0:1789302858.250204:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 11 pid: 6697 timeout: 795 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.250395:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 410 00000100:00100000:3.0:1789302858.250408:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350481536:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302858.250429:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302858.250459:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 11 pid: 6697 timeout: 795 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.250476:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 11 pid: 6697 timeout: 795 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.250525:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184f1c7 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c9c expref: 11 pid: 6697 timeout: 795 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.250549:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302858.250553:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302858.250563:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.250580:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.250630:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 10 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.250647:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371164800:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302858.250777:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0c00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302858.253279:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371164928:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302858.253373:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302858.254230:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e1180 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371164800:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302858.254260:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302858.254330:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350481536:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 3923us (7284us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302858.254341:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 410 00000100:00100000:2.0:1789302858.255007:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371164928 00000100:00100000:2.0:1789302858.255019:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371164928 00000100:00100000:2.0:1789302858.255148:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1789302858.255160:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3b2b9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371164928:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302858.255212:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 1703936 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302858.255298:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3b2b9c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371164928:12345-0@lo:13:osp-pre-1-0.0 Request processed in 141us (1925us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.255305:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:0.0:1789302858.255686:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371164928:0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1789302858.256011:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302858.262284:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.262396:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350481664 00000100:00100000:2.0:1789302858.262411:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350481664 00002000:00100000:2.0:1789302858.262479:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1c:0x0] (0->157) req@ffff960f41fbbb80 x1876219350481664/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:434/0 lens 648/0 e 0 to 0 dl 1789302869 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302858.262492:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41fbbb80 with x1876219350481664 ext(0->157) 00002000:00010000:2.0:1789302858.262499:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41fbbb80 00000100:00100000:2.0:1789302858.262562:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 120 00000100:00100000:2.0:1789302858.262573:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350481664:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:2.0:1789302858.266940:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbbb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350481664:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 4357us (4659us total) trans 4294967374 rc 0/0 00000100:00100000:2.0:1789302858.266954:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 120 00002000:00100000:2.0:1789302858.266967:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x1c:0x0] (0->157) req@ffff960f41fbbb80 x1876219350481664/t4294967374(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:434/0 lens 648/448 e 0 to 0 dl 1789302869 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302858.266978:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f41fbbb80 with x1876219350481664 ext(0->157) 00002000:00010000:2.0:1789302858.266984:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f41fbbb80 00000100:00100000:0.0:1789302858.279190:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.279408:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350481792 00000100:00100000:1.0:1789302858.279418:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350481792 00000100:00100000:1.0:1789302858.279488:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1368 00000100:00100000:1.0:1789302858.279499:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:31500:x1876219350481792:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:1.0:1789302858.348284:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968206 is committed 00000001:00080000:1.0:1789302858.348374:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968207 is committed 00000020:00080000:1.0:1789302858.348392:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 31 batchid updated 00000001:00080000:1.0:1789302858.348407:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968208 is committed 00000100:00100000:2.0:1789302858.348416:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31500:x1876219350481792:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 68913us (69229us total) trans 0 rc 0/0 00000020:00080000:1.0:1789302858.348419:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 31 00000020:00080000:1.0:1789302858.348431:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 31 00000100:00100000:2.0:1789302858.348433:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1368 00000100:00100000:1.0:1789302858.362790:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.363122:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350481920 00000100:00100000:0.0:1789302858.363132:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350481920 00000100:00100000:0.0:1789302858.363196:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 384 00000100:00100000:0.0:1789302858.363285:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350481920:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00010000:00010000:1.0:1789302858.397276:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302858.430420:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2345:x1876219350481920:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 67203us (67914us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302858.430433:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 384 00000001:00080000:2.0:1789302858.431667:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967374 is committed 00000100:00100000:0.0:1789302858.434543:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.434636:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350482176 00000100:00100000:2.0:1789302858.434645:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350482176 00000100:00100000:2.0:1789302858.434713:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1369 00000100:00100000:2.0:1789302858.434723:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31500:x1876219350482176:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302858.434780:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302858.434810:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1c0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c95 expref: 50 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.434832:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:2.0:1789302858.434838:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302858.434874:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.434888:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.434898:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302858.435216:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000040941519 reply state 00000000ece85ff4 transno 4294968209 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302858.435225:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302858.435311:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31500:x1876219350482176:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 588us (815us total) trans 4294968209 rc 0/0 00000100:00100000:2.0:1789302858.435321:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1369 00000100:00100000:3.0:1789302858.439815:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302858.439909:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.439923:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.439947:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.439954:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302858.439959:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.439967:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.439983:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:1.0:1789302858.439986:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302858.440226:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.440234:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f1d5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302858.443076:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350482304 00000100:00100000:0.0:1789302858.443095:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350482304 00000100:00100000:0.0:1789302858.443279:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1370 00000100:00100000:0.0:1789302858.443291:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31500:x1876219350482304:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302858.443318:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302858.443339:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 2/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0caa expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.443403:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 51 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.443416:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 51 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.443436:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302858.443440:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302858.443444:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345be00, rc 0) 00000100:00100000:0.0:1789302858.443540:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31500:x1876219350482304:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 251us (3734us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.443547:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1370 00000100:00100000:1.0:1789302858.450470:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.450625:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350482432 00000100:00100000:0.0:1789302858.450635:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350482432 00000100:00100000:0.0:1789302858.450704:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 385 00000100:00100000:0.0:1789302858.450715:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350482432:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:0.0:1789302858.451594:0:6697:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:0.0:1789302858.451600:0:6697:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb mtime from disk: 0 -> 1789302853 00002000:00010000:0.0:1789302858.451602:0:6697:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb atime from disk: 0 -> 1789302853 00002000:00010000:0.0:1789302858.451604:0:6697:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb ctime from disk: 0 -> 1789302853 00002000:00010000:0.0:1789302858.451607:0:6697:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1c:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:0.0:1789302858.451685:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350482432:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 969us (1221us total) trans 4294967375 rc 0/0 00000100:00100000:0.0:1789302858.451695:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 385 00000100:00100000:1.0:1789302858.458348:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.466068:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350482688 00000100:00100000:2.0:1789302858.466082:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350482688 00000100:00100000:3.0:1789302858.466471:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1371 00000100:00100000:3.0:1789302858.466482:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:31500:x1876219350482688:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:3.0:1789302858.466568:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.466586:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.466595:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.466641:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.466664:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 51 pid: 5844 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.466677:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371165056:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302858.466762:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03f940c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302858.476359:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.478469:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350482816 00000100:00100000:2.0:1789302858.478482:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350482816 00010000:00010000:2.0:1789302858.478580:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 52 pid: 5844 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302858.478623:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 411 00000100:00100000:2.0:1789302858.478631:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120bb850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350482816:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302858.478645:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302858.478662:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 52 pid: 5844 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.478675:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 52 pid: 5844 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.478696:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:2.0:1789302858.478701:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.478710:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.478718:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.478737:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302858.478794:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120bb850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350482816:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 163us (2444us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.478800:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 411 00010000:00010000:1.0:1789302858.482445:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f18f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x29:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c80 expref: 51 pid: 5844 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302858.482474:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2a00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371165056:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302858.482497:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484559:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484574:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484613:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484620:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484628:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484634:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302858.484638:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484645:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484678:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302858.484682:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1789302858.484687:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.484694:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f1e3 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302858.484815:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31500:x1876219350482688:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 18330us (26472us total) trans 0 rc -22/-22 00000100:00100000:0.0:1789302858.484825:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1371 00000100:00100000:0.0:1789302858.494965:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302858.500233:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350482944 00000100:00100000:3.0:1789302858.500249:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350482944 00000100:00100000:3.0:1789302858.500313:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 386 00000100:00100000:3.0:1789302858.500323:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350482944:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302858.500366:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.500385:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.500394:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302858.500398:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302858.500403:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.500440:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 11 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.500460:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 11 pid: 8571 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.500474:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371165184:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302858.500587:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03f94240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302858.504907:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.505557:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350483072 00000100:00100000:0.0:1789302858.505567:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350483072 00010000:00010000:0.0:1789302858.505609:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 12 pid: 8571 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302858.505662:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 412 00000100:00100000:0.0:1789302858.505671:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350483072:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302858.505685:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302858.505710:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 12 pid: 8571 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.505734:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 12 pid: 8571 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.505758:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302858.505764:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302858.505770:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.505780:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.505801:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302858.505960:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350483072:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 289us (1060us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.505968:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 412 00010000:00010000:3.0:1789302858.506410:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007030e32a/0x64fe43095184f1ce lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ca3 expref: 11 pid: 8571 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.506428:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371165184:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302858.506452:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506690:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506699:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506735:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506743:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506752:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506760:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302858.506765:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506774:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506787:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.506796:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000aedbc85e/0x64fe43095184f1ea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302858.506849:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350482944:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 6527us (11893us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.506858:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 386 00000100:00100000:1.0:1789302858.510226:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.510376:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350483328 00000100:00100000:2.0:1789302858.510387:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350483328 00000100:00100000:2.0:1789302858.510476:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1372 00000100:00100000:2.0:1789302858.510487:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:31500:x1876219350483328:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302858.510588:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.510605:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.510615:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.510836:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.510849:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 51 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.510857:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.510894:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 51 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.510909:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 51 pid: 18427 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302858.510924:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371165312:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302858.511015:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e0281b780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302858.514546:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.514645:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350483456 00000100:00100000:0.0:1789302858.514664:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350483456 00010000:00010000:0.0:1789302858.514700:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 52 pid: 18427 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302858.514744:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 413 00000100:00100000:0.0:1789302858.514754:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350483456:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302858.514768:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302858.514776:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 5/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 52 pid: 18427 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.514789:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 52 pid: 18427 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.514810:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302858.514815:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.514825:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.514832:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302858.514850:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302858.514899:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350483456:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 145us (361us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.514906:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 413 00010000:00010000:3.0:1789302858.515149:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f1dc lrc: 0/0,0 mode: --/CR res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0caa expref: 51 pid: 18427 timeout: 795 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302858.515166:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb8380 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371165312:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302858.515189:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.515198:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.515203:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:3.0:1789302858.515629:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968210 00000001:00080000:3.0:1789302858.515786:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968211 00000001:00080000:3.0:1789302858.515836:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968212 00000100:00100000:3.0:1789302858.515915:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371165440:0@lo:1000:mdt00_000.0 00000100:00100000:3.0:1789302858.515962:0:5843:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302858.516677:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371165440 00000100:00100000:0.0:1789302858.516684:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371165440 00000100:00100000:0.0:1789302858.516738:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 67 00000100:00100000:0.0:1789302858.516746:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371165440:12345-0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1789302858.516821:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371165440:12345-0@lo:1000:mdt00_000.0 Request processed in 76us (858us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302858.516829:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 67 00000100:00100000:3.0:1789302858.517719:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03f943c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302858.517753:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e2d80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371165440:0@lo:1000:mdt00_000.0 00000001:00080000:0.0:1789302858.519206:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968213 00000100:00100000:0.0:1789302858.519436:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0266d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371165568:0@lo:1000:mdt00_000.0 00000100:00100000:0.0:1789302858.519497:0:5843:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302858.519698:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdb3c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302858.519812:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371165568 00000100:00100000:2.0:1789302858.519820:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371165568 00000100:00100000:2.0:1789302858.519907:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 68 00000100:00100000:2.0:1789302858.519917:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5843:x1876219371165568:12345-0@lo:1000:mdt00_000.0 00000100:00100000:2.0:1789302858.520141:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5843:x1876219371165568:12345-0@lo:1000:mdt00_000.0 Request processed in 225us (643us total) trans 4294967376 rc 0/0 00000100:00100000:2.0:1789302858.520148:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 68 00000100:00100000:1.0:1789302858.520225:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0266d880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-mdtlov_UUID:5843:1876219371165568:0@lo:1000:mdt00_000.0 00000001:00080000:1.0:1789302858.520337:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968214 00010000:00010000:1.0:1789302858.520369:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520379:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520389:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520395:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302858.520399:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520406:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520422:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:1.0:1789302858.520426:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302858.520431:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520437:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2000/0x64fe43095184f1f8 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x29:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520467:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520473:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520480:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520485:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302858.520489:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520495:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520546:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:1.0:1789302858.520549:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302858.520556:0:5843:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 2/0,0 mode: --/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c6b expref: 101 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520568:0:5843:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c6b expref: 101 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520576:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c6b expref: 101 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.520604:0:5843:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2200/0x64fe43095184f12d lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0c6b expref: 101 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302858.520617:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef32a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371165696:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302858.520692:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e039ff540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302858.524616:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.525198:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef32a00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371165696:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302858.525226:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525241:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f126 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525274:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 3/0,1 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525287:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525297:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525318:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525325:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525331:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525336:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302858.525339:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525346:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525357:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:1.0:1789302858.525360:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302858.525364:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 2/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525370:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f1ff lrc: 0/0,0 mode: --/EX res: [0x200000402:0x28:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525380:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525387:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525392:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525397:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302858.525400:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525406:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525413:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x28:0x0].0x0 (ffff960f0cdf2a00) 00010000:00010000:1.0:1789302858.525415:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302858.525418:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.525424:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f1f1 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x28:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302858.525708:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:31500:x1876219350483328:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 15220us (15489us total) trans 4294968215 rc 0/0 00000100:00100000:1.0:1789302858.525718:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1372 00000020:00080000:1.0:1789302858.525736:0:5843:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:3.0:1789302858.526030:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302858.526035:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302858.526554:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1789302858.526563:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302858.526956:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350483584 00000100:00100000:1.0:1789302858.526965:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350483584 00000100:00100000:1.0:1789302858.527035:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:1.0:1789302858.527045:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e8850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:31499:x1876219350483584:12345-192.168.203.11@tcp:49:lseek_test.0 00000100:00100000:1.0:1789302858.527168:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e8850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:31499:x1876219350483584:12345-192.168.203.11@tcp:49:lseek_test.0 Request processed in 123us (2559us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302858.527175:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 277 00000100:00100000:1.0:1789302858.531746:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.531818:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350483712 00000100:00100000:2.0:1789302858.531829:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350483712 00000100:00100000:2.0:1789302858.531903:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 387 00000100:00100000:2.0:1789302858.531914:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+13:31499:x1876219350483712:12345-192.168.203.11@tcp:101:lseek_test.0 00010000:00010000:2.0:1789302858.531932:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302858.531955:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002a010bee/0x64fe43095184f206 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cb1 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302858.531981:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1c:0x0] initial LVB size: 158, mtime: 0x6aa69845, atime: 0x6aa69845, ctime: 0x6aa69845, blocks: 0xb 00010000:00010000:2.0:1789302858.532019:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=695s v=(0 0) 00010000:00010000:2.0:1789302858.532024:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302858.532032:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a010bee/0x64fe43095184f206 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cb1 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.532048:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002a010bee/0x64fe43095184f206 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cb1 expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302858.532060:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2000, rc 0) 00000100:00100000:2.0:1789302858.532118:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:31499:x1876219350483712:12345-192.168.203.11@tcp:101:lseek_test.0 Request processed in 206us (380us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.532125:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 387 00000100:00080000:1.0:1789302858.532415:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302858.534806:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302858.535642:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350483968 00000100:00100000:1.0:1789302858.535663:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350483968 00000100:00100000:1.0:1789302858.535719:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 278 00000100:00100000:1.0:1789302858.535741:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:31500:x1876219350483968:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:1.0:1789302858.536710:0:5847:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x29:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:1.0:1789302858.536827:0:5847:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968216 00010000:00010000:1.0:1789302858.536883:0:5847:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.536902:0:5847:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.536915:0:5847:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.536925:0:5847:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.536931:0:5847:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.536939:0:5847:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302858.536962:0:5847:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302858.537236:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302858.537243:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.537251:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302858.537255:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.537269:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.537282:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x29:0x0].0x0 (ffff960f2c94d200) 00010000:00010000:3.0:1789302858.537285:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302858.537290:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 1/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.537301:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f20d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x29:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5847 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302858.537317:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302858.537319:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302858.537321:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302858.538410:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:31500:x1876219350483968:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 2662us (3606us total) trans 4294968217 rc 0/0 00000100:00100000:0.0:1789302858.538421:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 278 00000020:00080000:0.0:1789302858.538429:0:5847:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:3.0:1789302858.542248:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302858.542385:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350484096 00000100:00100000:2.0:1789302858.542394:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350484096 00000100:00100000:2.0:1789302858.542453:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 388 00000100:00100000:2.0:1789302858.542462:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2346:x1876219350484096:12345-192.168.203.11@tcp:23:lseek_test.0 00000001:00080000:3.0:1789302858.573093:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967375 is committed 00000001:00080000:3.0:1789302858.573116:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967376 is committed 00000100:00100000:2.0:1789302858.578111:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2346:x1876219350484096:12345-192.168.203.11@tcp:23:lseek_test.0 Request processed in 35638us (35858us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302858.578126:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 388 00000100:00100000:3.0:1789302858.580876:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302858.581020:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350484352 00000100:00100000:0.0:1789302858.581030:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350484352 00000100:00100000:0.0:1789302858.581092:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 279 00000100:00100000:0.0:1789302858.581102:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:31499:x1876219350484352:12345-192.168.203.11@tcp:35:lseek_test.0 00000100:00100000:0.0:1789302858.581361:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:31499:x1876219350484352:12345-192.168.203.11@tcp:35:lseek_test.0 Request processed in 259us (493us total) trans 4294968218 rc 0/0 00000100:00100000:0.0:1789302858.581369:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 279 00000100:00100000:1.0:1789302858.764633:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302858.764667:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371165824:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302858.764741:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302858.764804:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371165824 00000100:00100000:1.0:1789302858.764810:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371165824 00000100:00100000:1.0:1789302858.764855:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 207 00000100:00100000:1.0:1789302858.764865:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371165824:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302858.764887:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3858759680 avail 3855613952 00000100:00100000:1.0:1789302858.764941:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371165824:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302858.764948:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 207 00000100:00100000:1.0:1789302858.764987:0:3302:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967376 gen 2 00000004:00080000:1.0:1789302858.764993:0:3302:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967376 committed transno 4294967376 00000100:00100000:1.0:1789302858.765019:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371165824:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302859.021605:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302859.021751:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350484608 00000100:00100000:0.0:1789302859.021762:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350484608 00000100:00100000:0.0:1789302859.021824:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 482 00000100:00100000:0.0:1789302859.021834:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2347:x1876219350484608:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302859.021925:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:2347:x1876219350484608:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 92us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302859.021933:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 482 00000100:00100000:3.0:1789302859.024702:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302859.026070:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350484480 00000100:00100000:1.0:1789302859.026082:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350484480 00000100:00100000:1.0:1789302859.026133:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 610 00000100:00100000:1.0:1789302859.026142:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350484480:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302859.026219:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350484480:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 78us (1524us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302859.026227:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 610 00000100:00100000:1.0:1789302859.026561:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302859.026582:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302859.026601:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350484864 00000100:00100000:1.0:1789302859.026605:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350484864 00000100:00100000:1.0:1789302859.026635:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350485120 00000100:00100000:1.0:1789302859.026637:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350485120 00000100:00100000:1.0:1789302859.026666:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:1.0:1789302859.026673:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350484864:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302859.026703:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350484864:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (144us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302859.026709:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:1.0:1789302859.026719:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 499 00000100:00100000:1.0:1789302859.026724:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350485120:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302859.026742:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350485120:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 18us (159us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302859.026747:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 499 00000100:00080000:1.0:1789302859.026760:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302859.030780:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302859.030843:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350484736 00000100:00100000:0.0:1789302859.030858:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350484736 00000100:00100000:0.0:1789302859.030926:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:0.0:1789302859.030949:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350484736:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302859.031029:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350484736:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 83us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302859.031036:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:1.0:1789302859.032554:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302859.032601:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302859.032791:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350484992 00000100:00100000:0.0:1789302859.032803:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350484992 00000100:00100000:0.0:1789302859.034376:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 483 00000100:00100000:0.0:1789302859.034397:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:2348:x1876219350484992:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302859.034518:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:2348:x1876219350484992:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 115us (1999us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302859.034530:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 483 00000100:00100000:2.0:1789302859.036203:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350485248 00000100:00100000:2.0:1789302859.036221:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350485248 00000100:00100000:2.0:1789302859.038950:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00100000:2.0:1789302859.038971:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350485248:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302859.039102:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350485248:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 137us (6499us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302859.039116:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00080000:3.0:1789302859.040291:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1789302859.418869:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302859.566125:0:21878:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302859.566135:0:21878:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302859.566184:0:21878:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 41 count 4 00000040:00080000:3.0:1789302859.566190:0:21878:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:1.0:1789302859.889159:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967376 is committed 00000001:00080000:1.0:1789302859.889178:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967374 is committed 00010000:00010000:1.0:1789302860.428947:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302860.560962:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302860.560973:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302860.561295:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 41 count 3 00000040:00080000:2.0:1789302860.561302:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302861.453783:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302862.480820:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302862.861865:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371165952:0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.861934:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302862.862087:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371165952 00000100:00100000:0.0:1789302862.862096:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371165952 00000100:00100000:0.0:1789302862.862144:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 611 00000100:00100000:0.0:1789302862.862152:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371165952:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302862.862303:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371166208:0@lo:400:kworker.0 00000100:00100000:3.0:1789302862.862346:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302862.862370:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302862.862375:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371166336:0@lo:400:kworker.0 00000100:00100000:3.0:1789302862.862401:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302862.862430:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371166208 00000100:00100000:3.0:1789302862.862434:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371166208 00000100:00100000:3.0:1789302862.862471:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789302862.862478:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371166208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302862.862537:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371166208:12345-0@lo:400:kworker.0 Request processed in 59us (188us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302862.862545:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:3.0:1789302862.862567:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371166336 00000100:00100000:3.0:1789302862.862570:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371166336 00000100:00100000:3.0:1789302862.862596:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:3.0:1789302862.862602:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371166336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302862.862643:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859ac50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371166336:12345-0@lo:400:kworker.0 Request processed in 41us (241us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302862.862648:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00080000:3.0:1789302862.862797:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302862.862802:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302862.862832:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371166336:0@lo:400:kworker.0 00000100:00100000:3.0:1789302862.862842:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371166208:0@lo:400:kworker.0 00000100:00100000:0.0:1789302862.863059:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371165952:12345-0@lo:400:kworker.0 Request processed in 905us (1123us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302862.863066:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 611 00000100:00100000:2.0:1789302862.863249:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371165952:0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.863265:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371166464:0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.863293:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302862.863390:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371166464 00000100:00100000:3.0:1789302862.863394:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371166464 00000100:00100000:2.0:1789302862.863462:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1789302862.863468:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371166080:0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.863491:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302862.863525:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371166592:0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.863547:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302862.863576:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371166080 00000100:00100000:2.0:1789302862.863580:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371166080 00000100:00100000:2.0:1789302862.863610:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:2.0:1789302862.863630:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371166080:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.863677:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371166080:12345-0@lo:400:kworker.0 Request processed in 51us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302862.863683:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:2.0:1789302862.863712:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371166592 00000100:00100000:2.0:1789302862.863716:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371166592 00000100:00100000:3.0:1789302862.863777:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:3.0:1789302862.863784:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371166464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302862.863813:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371166464:12345-0@lo:400:kworker.0 Request processed in 30us (520us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302862.863819:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00080000:0.0:1789302862.864185:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302862.864190:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302862.864278:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1789302862.864286:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371166592:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.864331:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371166592:12345-0@lo:400:kworker.0 Request processed in 46us (783us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302862.864337:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1789302862.864369:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371166080:0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.864377:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967376 gen 2 00000004:00080000:2.0:1789302862.864382:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967375 committed transno 4294967376 00000004:00080000:2.0:1789302862.864396:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f11de5880, transno 4294967376 00000100:00100000:2.0:1789302862.864408:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371166592:0@lo:400:kworker.0 00000100:00100000:2.0:1789302862.864419:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371166464:0@lo:400:kworker.0 00000100:00100000:3.0:1789302863.373104:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302863.373127:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371166720:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302863.373193:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302863.373276:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371166720 00000100:00100000:3.0:1789302863.373283:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371166720 00000100:00100000:3.0:1789302863.373325:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1789302863.373341:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371166720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302863.373377:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302863.373422:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371166720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 88us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302863.373430:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1789302863.373471:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371166720:0@lo:13:osp-pre-1-0.0 00000001:00080000:2.0:1789302863.456233:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968218 is committed 00000001:00080000:2.0:1789302863.456252:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968214 is committed 00000020:00080000:2.0:1789302863.456258:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f03f943c0 refcount 2 committed 0 result 0 batchid 32 00000020:00080000:2.0:1789302863.456262:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3feebe00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f03f940c0 00000020:00080000:2.0:1789302863.456266:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f3feeb900 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f1daf6c00 00000020:00080000:2.0:1789302863.456303:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 32 committed batchid 31 00000020:00080000:2.0:1789302863.456574:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 31 00000001:00080000:2.0:1789302863.457950:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968217 is committed 00010000:00010000:1.0:1789302863.500725:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302863.889831:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302863.889853:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371166848:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302863.889930:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302863.889999:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371166848 00000100:00100000:1.0:1789302863.890007:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371166848 00000100:00100000:1.0:1789302863.890054:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789302863.890062:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371166848:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302863.890094:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302863.890137:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371166848:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302863.890143:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 209 00000100:00100000:1.0:1789302863.890199:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c6680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371166848:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302864.140636:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302864.140679:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302864.140721:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302864.140743:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302864.140767:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302864.140786:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302864.140810:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302864.140843:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350485888 00000100:00100000:1.0:1789302864.140856:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350485888 00000100:00100000:0.0:1789302864.140864:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350485632 00000100:00100000:0.0:1789302864.140869:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350485632 00000100:00100000:1.0:1789302864.140921:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350485504 00000100:00100000:0.0:1789302864.140921:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350486144 00000100:00100000:1.0:1789302864.140924:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350485504 00000100:00100000:0.0:1789302864.140924:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350486144 00000100:00100000:0.0:1789302864.140943:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350485760 00000100:00100000:0.0:1789302864.140945:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350485760 00000100:00100000:1.0:1789302864.140948:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00100000:0.0:1789302864.140958:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350486016 00000100:00100000:1.0:1789302864.140959:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2345:x1876219350485888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302864.140960:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350486016 00000100:00100000:0.0:1789302864.140973:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 504 00000100:00100000:0.0:1789302864.140979:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2345:x1876219350485632:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302864.141050:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2345:x1876219350485632:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 70us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302864.141059:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 504 00000100:00100000:0.0:1789302864.141078:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 505 00000100:00100000:0.0:1789302864.141084:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350486144:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302864.141104:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350485376 00000100:00100000:3.0:1789302864.141109:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350485376 00000100:00100000:3.0:1789302864.141156:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:3.0:1789302864.141177:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350485376:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302864.142861:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2345:x1876219350485888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1883us (2119us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302864.142879:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00100000:1.0:1789302864.142923:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:1.0:1789302864.142938:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:2345:x1876219350485504:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302864.142999:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:2345:x1876219350485504:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 67us (2231us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302864.143017:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:1.0:1789302864.143052:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00100000:1.0:1789302864.143062:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350485760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302864.143088:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350485760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (2301us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302864.143094:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00100000:1.0:1789302864.143110:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 507 00000100:00100000:1.0:1789302864.143116:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350486016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302864.143138:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350486016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (2327us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302864.143143:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 507 00000100:00100000:3.0:1789302864.146838:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350485376:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5669us (6211us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302864.146858:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:0.0:1789302864.147105:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350486144:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5343us (5684us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302864.147120:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 505 00000100:00080000:3.0:1789302864.148081:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302864.148089:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302864.148094:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302864.525877:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302864.976118:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302864.976223:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350486272 00000100:00100000:2.0:1789302864.976235:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350486272 00000100:00100000:2.0:1789302864.976301:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1373 00000100:00100000:2.0:1789302864.976311:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31471:x1876219350486272:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:2.0:1789302864.976423:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:31471:x1876219350486272:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 113us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302864.976430:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1373 00010000:00010000:3.0:1789302865.548778:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302866.574020:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:1.0:1789302866.794071:0:22077:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12s: race between restore requests ==== 08:34:25 (1789302865) 00000100:00100000:1.0:1789302867.221140:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.221245:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350486400 00000100:00100000:2.0:1789302867.221256:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350486400 00000100:00100000:2.0:1789302867.221328:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1374 00000100:00100000:2.0:1789302867.221341:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:32115:x1876219350486400:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:2.0:1789302867.221455:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:32115:x1876219350486400:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 118us (326us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302867.221463:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1374 00000100:00080000:3.0:1789302867.221532:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302867.251485:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.251575:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350486528 00000100:00100000:1.0:1789302867.251595:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350486528 00000100:00100000:1.0:1789302867.251674:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1375 00000100:00100000:1.0:1789302867.251684:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:32116:x1876219350486528:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302867.251705:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.251726:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f214 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cb8 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.251808:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.251824:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.251836:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.251898:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252036:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252044:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252271:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252279:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252286:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252291:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.252295:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302867.252410:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302867.252547:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252561:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302867.252565:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.252570:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252577:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f21b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252600:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252606:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252612:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252617:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.252620:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252627:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252635:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff960f2c94d100) 00010000:00010000:1.0:1789302867.252637:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.252640:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252650:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f222 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252682:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f214 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cb8 expref: 103 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252692:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302867.252695:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.252698:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f214 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cb8 expref: 103 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.252707:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9e00, rc 0) 00000100:00100000:1.0:1789302867.253059:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:32116:x1876219350486528:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1373us (1581us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302867.253067:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1375 00000100:00100000:0.0:1789302867.260236:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.260403:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350486784 00000100:00100000:1.0:1789302867.260414:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350486784 00000100:00100000:1.0:1789302867.260479:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1376 00000100:00100000:1.0:1789302867.260490:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:32116:x1876219350486784:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:1.0:1789302867.260611:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302867.260635:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184ef7b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0bd1 expref: 102 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.260653:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302867.260667:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302867.260874:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.260888:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.260897:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.260928:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 47 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.260941:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 47 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302867.260954:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371166976:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302867.261412:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33411f00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302867.264917:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.265061:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350486912 00000100:00100000:2.0:1789302867.265073:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350486912 00010000:00010000:2.0:1789302867.265114:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 48 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302867.265156:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 414 00000100:00100000:2.0:1789302867.265167:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350486912:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302867.265182:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302867.265189:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 48 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.265203:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 48 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.265223:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:2.0:1789302867.265229:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.265243:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.265251:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.265287:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302867.265349:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:2348:x1876219350486912:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 183us (440us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302867.265356:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 414 00010000:00010000:1.0:1789302867.265975:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184efba lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0be6 expref: 47 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302867.265994:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371166976:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302867.266016:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.266026:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.266032:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.266054:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.266064:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.266071:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302867.268212:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 16384 00010000:00010000:1.0:1789302867.268234:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.268245:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.268267:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.268274:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.268279:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.268287:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.268304:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302867.268307:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.268312:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.268318:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f229 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302867.268343:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e8bde5ca reply state 0000000023b70799 transno 4294968219 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302867.268352:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302867.268455:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:32116:x1876219350486784:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 7964us (8225us total) trans 4294968219 rc 0/0 00000100:00100000:1.0:1789302867.268464:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1376 00010000:00010000:2.0:1789302867.273131:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.273148:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.273156:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.273162:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302867.273168:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.273175:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.273191:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff960f2c94d100) 00010000:00010000:2.0:1789302867.273195:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302867.273200:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.273211:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f230 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302867.281323:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.281434:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487040 00000100:00100000:1.0:1789302867.281443:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487040 00000100:00100000:1.0:1789302867.281546:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1377 00000100:00100000:1.0:1789302867.281558:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+102:32117:x1876219350487040:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302867.281583:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.281616:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f237 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cbf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.281685:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f23e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.281699:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f23e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.281709:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f23e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302867.281743:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:1.0:1789302867.281747:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302867.281766:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f23e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.281794:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f237 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cbf expref: 104 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.281812:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f23e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cbf expref: 103 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.281821:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302867.281824:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302867.281827:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc 0) 00000100:00100000:1.0:1789302867.281888:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:32117:x1876219350487040:12345-192.168.203.11@tcp:101:bash.0 Request processed in 333us (573us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.281894:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1377 00000100:00100000:3.0:1789302867.285720:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.285867:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487168 00000100:00100000:1.0:1789302867.285878:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487168 00000100:00100000:1.0:1789302867.285939:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1378 00000100:00100000:1.0:1789302867.285950:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:32117:x1876219350487168:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302867.285970:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.285991:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f245 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cc6 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286062:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286077:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286088:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286104:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286110:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286116:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286292:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f25a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286303:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f25a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286309:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f25a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302867.286336:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 16384 00000004:00200000:1.0:1789302867.286340:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302867.286356:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f25a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286368:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286374:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286381:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286390:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.286394:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286401:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286412:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302867.286415:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.286419:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286425:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f24c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286438:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286443:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286449:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286454:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.286456:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286462:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286474:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff960f2c94d500) 00010000:00010000:1.0:1789302867.286476:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.286479:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286485:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f253 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286546:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f245 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cc6 expref: 105 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286561:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8400/0x64fe43095184f25a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cc6 expref: 104 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.286568:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.286570:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302867.286572:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8400, rc 0) 00000100:00100000:1.0:1789302867.286639:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32117:x1876219350487168:12345-192.168.203.11@tcp:101:bash.0 Request processed in 691us (928us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.286646:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1378 00000100:00100000:3.0:1789302867.290130:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.290215:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487296 00000100:00100000:1.0:1789302867.290225:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487296 00000100:00100000:1.0:1789302867.290296:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1379 00000100:00100000:1.0:1789302867.290307:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32117:x1876219350487296:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:1.0:1789302867.290328:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.290349:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f261 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ccd expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290424:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290438:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290449:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290558:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290567:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290574:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290647:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290654:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290661:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290674:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.290679:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290691:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290704:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.290707:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.290713:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290720:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f268 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290733:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290739:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290745:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290750:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.290753:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290758:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290765:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f2c94df00) 00010000:00010000:1.0:1789302867.290767:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.290770:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290777:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f26f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290813:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f261 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ccd expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290823:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.290825:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.290828:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f261 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ccd expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.290838:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9e00, rc 0) 00000100:00100000:1.0:1789302867.290896:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32117:x1876219350487296:12345-192.168.203.11@tcp:101:bash.0 Request processed in 590us (775us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302867.290902:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1379 00000100:00100000:0.0:1789302867.306307:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.306467:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487424 00000100:00100000:2.0:1789302867.306479:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487424 00000100:00100000:2.0:1789302867.306571:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1380 00000100:00100000:2.0:1789302867.306582:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+47:32118:x1876219350487424:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302867.306614:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302867.306636:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f276 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cd4 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.306712:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.306727:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.306738:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302867.306775:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302867.306778:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302867.306795:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.306818:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f276 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 49 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.306838:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.306847:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:2.0:1789302867.306851:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302867.306854:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:2.0:1789302867.306914:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266f100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:32118:x1876219350487424:12345-192.168.203.11@tcp:101:rm.0 Request processed in 333us (615us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302867.306920:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1380 00000100:00100000:0.0:1789302867.310434:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.310553:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487552 00000100:00100000:2.0:1789302867.310564:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487552 00000100:00100000:2.0:1789302867.310637:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1381 00000100:00100000:2.0:1789302867.310648:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:32118:x1876219350487552:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302867.310679:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302867.310701:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f284 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cdb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310766:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310782:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310793:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310807:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310814:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310824:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310916:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184f299 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310924:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184f299 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310932:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184f299 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302867.310959:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 16384 00000004:00200000:2.0:1789302867.310962:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302867.310980:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184f299 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310992:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.310999:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311006:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311012:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302867.311015:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311023:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311035:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:2.0:1789302867.311038:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302867.311042:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311047:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f28b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311061:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311067:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311071:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311076:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302867.311078:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311083:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311090:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x81f12197 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302867.311092:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302867.311095:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311100:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f292 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x81f12197 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311134:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f284 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cdb expref: 50 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311147:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbee00/0x64fe43095184f299 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cdb expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.311155:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:2.0:1789302867.311157:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302867.311159:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbee00, rc 0) 00000100:00100000:2.0:1789302867.311212:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0266d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32118:x1876219350487552:12345-192.168.203.11@tcp:101:rm.0 Request processed in 566us (788us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302867.311220:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1381 00000100:00100000:3.0:1789302867.314988:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.315100:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487680 00000100:00100000:1.0:1789302867.315111:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487680 00000100:00100000:1.0:1789302867.315168:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1382 00000100:00100000:1.0:1789302867.315179:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32118:x1876219350487680:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302867.315198:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.315221:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f2a0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ce2 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315288:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315302:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315314:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315328:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315335:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315341:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315402:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315409:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315415:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315422:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.315426:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315434:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315445:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.315448:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.315453:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315459:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f2a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315473:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315479:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315485:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315489:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.315492:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315497:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315555:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f2c94df00) 00010000:00010000:1.0:1789302867.315558:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.315561:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315566:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f2ae lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315613:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f2a0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ce2 expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315628:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.315630:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.315634:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f2a0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ce2 expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.315645:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9e00, rc 0) 00000100:00100000:1.0:1789302867.315707:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32118:x1876219350487680:12345-192.168.203.11@tcp:101:rm.0 Request processed in 530us (729us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302867.315714:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1382 00000100:00100000:0.0:1789302867.319951:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.320030:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487808 00000100:00100000:1.0:1789302867.320043:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487808 00000100:00100000:1.0:1789302867.320104:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1383 00000100:00100000:1.0:1789302867.320115:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32118:x1876219350487808:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302867.320138:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.320161:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f2b5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ce9 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320234:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320249:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320262:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320274:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320281:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320287:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320345:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320352:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320361:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320366:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.320370:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320379:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320390:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.320394:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.320399:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320405:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2bc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320419:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320425:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320430:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320434:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.320436:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320442:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320448:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f2c94d500) 00010000:00010000:1.0:1789302867.320449:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.320452:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320456:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2c3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320492:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f2b5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ce9 expref: 50 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320560:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.320563:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.320567:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f2b5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ce9 expref: 50 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.320583:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8800, rc 0) 00000100:00100000:1.0:1789302867.320653:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32118:x1876219350487808:12345-192.168.203.11@tcp:101:rm.0 Request processed in 540us (710us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302867.320672:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1383 00000100:00100000:0.0:1789302867.335718:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.335823:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350487936 00000100:00100000:1.0:1789302867.335833:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350487936 00000100:00100000:1.0:1789302867.335896:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1384 00000100:00100000:1.0:1789302867.335907:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32119:x1876219350487936:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302867.335929:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.335952:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f2ca lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cf0 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336020:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336035:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336046:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336061:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336068:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336075:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336128:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336135:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336142:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336148:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.336153:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336161:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336172:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.336175:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.336180:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336186:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f2d1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336202:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336207:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336213:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336218:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.336221:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336228:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336234:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f2c94d500) 00010000:00010000:1.0:1789302867.336237:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.336240:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336245:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f2d8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336295:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f2ca lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cf0 expref: 50 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336306:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.336308:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.336311:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f2ca lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cf0 expref: 50 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.336320:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8800, rc 0) 00000100:00100000:1.0:1789302867.336383:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32119:x1876219350487936:12345-192.168.203.11@tcp:101:cp.0 Request processed in 477us (673us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302867.336390:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1384 00000100:00100000:0.0:1789302867.341144:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.341249:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350488064 00000100:00100000:2.0:1789302867.341391:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350488064 00000100:00100000:2.0:1789302867.341459:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1385 00000100:00100000:2.0:1789302867.341470:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32119:x1876219350488064:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302867.341491:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302867.341549:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f2df lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cf7 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341631:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341646:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341667:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341684:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341693:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341699:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341752:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341760:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341766:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341772:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302867.341776:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341783:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341794:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:2.0:1789302867.341797:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302867.341802:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341809:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f2e6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341822:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341828:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341833:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341839:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302867.341842:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341848:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341855:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f10d3cc00) 00010000:00010000:2.0:1789302867.341858:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302867.341861:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341866:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2ed lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341898:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f2df lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cf7 expref: 50 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341908:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:2.0:1789302867.341911:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302867.341915:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f2df lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cf7 expref: 50 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.341925:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbe400, rc 0) 00000100:00100000:2.0:1789302867.341978:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32119:x1876219350488064:12345-192.168.203.11@tcp:101:cp.0 Request processed in 510us (844us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302867.341985:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1385 00000100:00100000:0.0:1789302867.349764:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.349872:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350488320 00000100:00100000:2.0:1789302867.349883:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350488320 00000100:00100000:2.0:1789302867.349947:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1386 00000100:00100000:2.0:1789302867.349958:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32119:x1876219350488320:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302867.349980:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302867.350002:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f2f4 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0cfe expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.350123:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.350137:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.350149:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.350163:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.350171:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.350177:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302867.350909:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1c:0x0], index 1: 0 00010000:00010000:2.0:1789302867.351094:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351105:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351114:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351149:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351157:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351163:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351170:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302867.351174:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351183:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351196:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:2.0:1789302867.351199:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302867.351205:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351216:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f2fb lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302867.351230:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008828c188 reply state 00000000a20102e0 transno 4294968220 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302867.351238:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302867.351268:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f2f4 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 52 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351277:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.351282:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:2.0:1789302867.351284:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302867.351286:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302867.351347:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32119:x1876219350488320:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1393us (1595us total) trans 4294968220 rc 0/0 00000100:00100000:2.0:1789302867.351353:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1386 00010000:00010000:1.0:1789302867.353746:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.353761:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.353769:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.353775:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302867.353782:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.353790:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.353806:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f10d3cc00) 00010000:00010000:1.0:1789302867.353810:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302867.353814:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.353821:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f302 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302867.357071:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.357152:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350488448 00000100:00100000:1.0:1789302867.357163:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350488448 00000100:00100000:1.0:1789302867.357249:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 389 00000100:00100000:1.0:1789302867.357260:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350488448:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302867.357278:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.357299:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d05 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302867.357332:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1c:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789302867.357354:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=704s v=(0 0) 00010000:00010000:1.0:1789302867.357359:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302867.357368:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.357388:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 10 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.357402:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8800, rc 0) 00000100:00100000:1.0:1789302867.357460:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350488448:12345-192.168.203.11@tcp:101:cp.0 Request processed in 201us (398us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.357467:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 389 00000100:00080000:3.0:1789302867.357536:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302867.362649:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302867.363159:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350488576 00000100:00100000:3.0:1789302867.363169:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350488576 00000100:00100000:3.0:1789302867.363329:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:3.0:1789302867.363340:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:32119:x1876219350488576:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302867.363357:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302867.363377:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d0c expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.363409:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.363417:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302867.363421:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302867.363426:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.363460:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 11 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.363473:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 11 pid: 18578 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302867.363487:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371167104:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302867.363637:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03f94540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302867.366714:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.366849:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350488704 00000100:00100000:2.0:1789302867.366858:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350488704 00010000:00010000:2.0:1789302867.366896:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 12 pid: 18578 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302867.366940:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 415 00000100:00100000:2.0:1789302867.366950:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350488704:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302867.366965:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302867.366987:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 12 pid: 18578 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.367000:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 12 pid: 18578 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.367025:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302867.367028:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302867.367036:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 12 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.367046:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 12 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.367080:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 12 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302867.367093:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371167232:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302867.367169:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302867.369601:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb8380 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0001_UUID:5829:1876219371167232:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302867.369626:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302867.369731:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350488704:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 2781us (3026us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302867.369741:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 415 00010000:00010000:1.0:1789302867.371014:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007030e32a/0x64fe43095184f317 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d05 expref: 11 pid: 18578 timeout: 804 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302867.371036:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371167104:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302867.371061:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.371076:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:1.0:1789302867.371132:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:32119:x1876219350488576:12345-192.168.203.11@tcp:101:cp.0 Request processed in 7794us (8493us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.371143:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 390 00000100:00100000:2.0:1789302867.374698:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302867.374840:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350488960 00000100:00100000:0.0:1789302867.374851:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350488960 00000100:00100000:0.0:1789302867.374913:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 280 00000100:00100000:0.0:1789302867.374923:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32119:x1876219350488960:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302867.375339:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32119:x1876219350488960:12345-192.168.203.11@tcp:35:cp.0 Request processed in 416us (648us total) trans 4294968221 rc 0/0 00000100:00100000:0.0:1789302867.375348:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 280 00000100:00100000:2.0:1789302867.432943:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302867.433049:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350489088 00000100:00100000:3.0:1789302867.433060:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350489088 00000100:00100000:3.0:1789302867.433129:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1387 00000100:00100000:3.0:1789302867.433139:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32122:x1876219350489088:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302867.433168:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302867.433192:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f325 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d13 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433274:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433291:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433303:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433319:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433326:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433332:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433431:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433439:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433446:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302867.433478:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00000004:00200000:3.0:1789302867.433483:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302867.433545:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433558:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433566:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433573:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433579:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433595:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302867.433600:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433608:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433619:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:3.0:1789302867.433623:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302867.433627:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433633:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f32c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433648:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433659:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433664:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433669:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302867.433671:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433677:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433683:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302867.433686:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302867.433688:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433693:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184f333 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433722:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f325 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 106 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433735:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 105 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.433744:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:3.0:1789302867.433746:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302867.433749:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302867.434476:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32122:x1876219350489088:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1333us (1536us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302867.434486:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1387 00000100:00100000:2.0:1789302867.443232:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302867.443423:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350489216 00000100:00100000:0.0:1789302867.443435:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350489216 00000100:00100000:0.0:1789302867.443563:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 391 00000100:00100000:0.0:1789302867.443574:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350489216:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302867.443607:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302867.443625:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df8c373b/0x64fe43095184f341 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d1a expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302867.443863:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371167360:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302867.443966:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4240 going to sleep for 11 seconds 00000100:00080000:3.0:1789302867.445633:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:0.0:1789302867.447747:0:13439:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1789302867.447755:0:13439:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb mtime: 0 -> 1789302867 00002000:00010000:0.0:1789302867.447758:0:13439:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb atime: 0 -> 1789302867 00002000:00010000:0.0:1789302867.447760:0:13439:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb ctime: 0 -> 1789302867 00000100:00100000:0.0:1789302867.447784:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70fb80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371167360:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302867.447822:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df8c373b/0x64fe43095184f341 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d1a expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.447844:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df8c373b/0x64fe43095184f341 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d1a expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.447859:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc 0) 00000100:00100000:0.0:1789302867.447920:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350489216:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4347us (4826us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302867.447929:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 391 00000100:00100000:2.0:1789302867.452893:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.453198:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350489344 00000100:00100000:1.0:1789302867.453209:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350489344 00000100:00100000:1.0:1789302867.453280:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1388 00000100:00100000:1.0:1789302867.453293:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32122:x1876219350489344:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302867.453314:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.453334:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f348 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d21 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.453405:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.453419:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.453431:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302867.453469:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x4f:0x0]: returning size 16384 00000004:00200000:1.0:1789302867.453472:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302867.453488:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.454550:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f348 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 107 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.454576:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 106 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.454599:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:1.0:1789302867.454602:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302867.454605:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8000, rc 0) 00000100:00100000:1.0:1789302867.454727:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32122:x1876219350489344:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1435us (1841us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.454735:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1388 00000100:00100000:2.0:1789302867.457684:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.457816:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350489472 00000100:00100000:1.0:1789302867.457827:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350489472 00000100:00100000:1.0:1789302867.457886:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 392 00000100:00100000:1.0:1789302867.457897:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350489472:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302867.457916:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.457934:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b6351f7f/0x64fe43095184f356 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d28 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302867.457995:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371167488:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302867.458082:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33411f00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302867.470631:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371167488:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302867.470690:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b6351f7f/0x64fe43095184f356 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d28 expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.470717:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b6351f7f/0x64fe43095184f356 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d28 expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.470732:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9e00, rc 0) 00000100:00100000:1.0:1789302867.471047:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350489472:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 13150us (13371us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302867.471057:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 392 00000100:00100000:0.0:1789302867.482939:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.483029:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350489728 00000100:00100000:1.0:1789302867.483039:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350489728 00000100:00100000:1.0:1789302867.483111:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1389 00000100:00100000:1.0:1789302867.483122:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32122:x1876219350489728:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:1.0:1789302867.483411:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].41, 136 off13768 00000100:00100000:1.0:1789302867.483539:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32122:x1876219350489728:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 418us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.483548:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1389 00010000:00010000:2.0:1789302867.598469:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302867.726894:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302867.726904:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302867.726947:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 42 count 4 00000040:00080000:1.0:1789302867.726954:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302867.727235:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371167616:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302867.727339:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f334119c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302867.728156:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.728310:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302867.728638:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350489856 00000100:00100000:3.0:1789302867.728646:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350489856 00002000:00100000:3.0:1789302867.728700:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1c:0x0] (0->157) req@ffff960e0114ad80 x1876219350489856/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:443/0 lens 648/0 e 0 to 0 dl 1789302878 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302867.728711:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0114ad80 with x1876219350489856 ext(0->157) 00002000:00010000:3.0:1789302867.728718:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0114ad80 00000100:00100000:3.0:1789302867.728753:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 121 00000100:00100000:3.0:1789302867.728762:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350489856:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:3.0:1789302867.729208:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350489856:12345-192.168.203.11@tcp:4:cp.0 Request processed in 446us (897us total) trans 4294967377 rc 0/0 00000100:00100000:3.0:1789302867.729217:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 121 00002000:00100000:3.0:1789302867.729224:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1c:0x0] (0->157) req@ffff960e0114ad80 x1876219350489856/t4294967377(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:443/0 lens 648/448 e 0 to 0 dl 1789302878 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302867.729233:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0114ad80 with x1876219350489856 ext(0->157) 00002000:00010000:3.0:1789302867.729236:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0114ad80 00000100:00100000:3.0:1789302867.729317:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350489984 00000100:00100000:3.0:1789302867.729322:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350489984 00000100:00100000:3.0:1789302867.729351:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 393 00000100:00100000:3.0:1789302867.729358:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350489984:12345-192.168.203.11@tcp:16:kworker.0 00000100:00100000:3.0:1789302867.737048:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371167616:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302867.739874:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.739923:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350490112 00000100:00100000:2.0:1789302867.739933:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350490112 00000100:00100000:2.0:1789302867.739988:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1390 00000100:00100000:2.0:1789302867.739999:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32132:x1876219350490112:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302867.740034:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302867.740126:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32132:x1876219350490112:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 129us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302867.740134:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1390 00000100:00100000:2.0:1789302867.764011:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.764730:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350490368 00000100:00100000:1.0:1789302867.764752:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350490368 00000100:00100000:1.0:1789302867.764812:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1391 00000100:00100000:1.0:1789302867.764823:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32132:x1876219350490368:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302867.764844:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.764864:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f35d lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d2f expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.764963:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f364 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.764981:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f364 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.764994:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f364 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302867.765005:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:1.0:1789302867.765149:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f35d lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d2f expref: 55 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.765165:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f364 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d2f expref: 54 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.765173:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:1.0:1789302867.765176:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302867.765180:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8200, rc 0) 00000100:00100000:1.0:1789302867.765238:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32132:x1876219350490368:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 418us (1236us total) trans 4294968222 rc 0/0 00000100:00100000:1.0:1789302867.765466:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1391 00000100:00100000:1.0:1789302867.786880:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.786920:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.786955:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350490624 00000100:00100000:1.0:1789302867.786965:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350490624 00000100:00100000:1.0:1789302867.787425:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 64 00000100:00100000:1.0:1789302867.787437:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2347:x1876219350490624:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302867.787456:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302867.787482:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f364 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d2f expref: 54 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.787528:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:1.0:1789302867.787536:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302867.787598:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2347:x1876219350490624:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 164us (676us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.787605:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 64 00000100:00100000:0.0:1789302867.789178:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350490496 00000100:00100000:0.0:1789302867.789191:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350490496 00000100:00100000:0.0:1789302867.789248:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 394 00000100:00100000:0.0:1789302867.789258:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350490496:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302867.789301:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.789320:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 13 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.789329:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302867.789333:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302867.789337:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.789371:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 13 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.789383:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 13 pid: 6695 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302867.789397:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371167744:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302867.789479:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4240 going to sleep for 11 seconds 00000001:00080000:1.0:1789302867.795381:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967377 is committed 00000100:00100000:3.0:1789302867.795605:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350489984:12345-192.168.203.11@tcp:16:kworker.0 Request processed in 66241us (67445us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302867.795616:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 393 00000100:00100000:2.0:1789302867.797627:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371167744:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302867.797660:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302867.803393:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302867.803700:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350490752 00000100:00100000:3.0:1789302867.803711:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350490752 00010000:00010000:3.0:1789302867.803759:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 11 pid: 6695 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302867.803807:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 416 00000100:00100000:3.0:1789302867.803818:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350490752:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302867.803833:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00002000:00010000:3.0:1789302867.803853:0:8463:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x1c:0x0] updating lvb blocks from disk: 1 -> 11 00010000:00010000:3.0:1789302867.803860:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 11 pid: 6695 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.803876:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 11 pid: 6695 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.803900:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f31e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d0c expref: 11 pid: 6695 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.803916:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302867.803920:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302867.803926:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.803943:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.803977:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302867.804032:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350490752:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 216us (657us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302867.804041:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 416 00010000:00010000:3.0:1789302867.804070:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804082:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804110:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804134:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804145:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804153:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302867.804158:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804175:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804187:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.804196:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184f372 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302867.804261:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350490496:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 15003us (17392us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302867.804269:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 394 00000100:00100000:1.0:1789302867.818391:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302867.818474:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350491008 00000100:00100000:3.0:1789302867.818484:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350491008 00000100:00100000:3.0:1789302867.818570:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1392 00000100:00100000:3.0:1789302867.818591:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32132:x1876219350491008:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302867.818713:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32132:x1876219350491008:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 133us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302867.818721:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1392 00000100:00100000:2.0:1789302867.843601:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302867.843782:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350491136 00000100:00100000:3.0:1789302867.843793:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350491136 00000100:00100000:3.0:1789302867.843856:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 395 00000100:00100000:3.0:1789302867.843868:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350491136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302867.843885:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302867.843907:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d36 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302867.843934:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1c:0x0] initial LVB size: 158, mtime: 0x6aa69853, atime: 0x0, ctime: 0x6aa69853, blocks: 0xb 00010000:00010000:3.0:1789302867.843953:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=704s v=(0 0) 00010000:00010000:3.0:1789302867.843957:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302867.843967:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 10 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.843984:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 10 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302867.843997:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302867.844068:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350491136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 203us (485us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302867.844076:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 395 00000100:00100000:3.0:1789302867.848157:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.849895:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350491392 00000100:00100000:2.0:1789302867.849920:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350491392 00000100:00100000:2.0:1789302867.849981:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1393 00000100:00100000:2.0:1789302867.849992:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32132:x1876219350491392:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302867.850107:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32132:x1876219350491392:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 117us (1958us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302867.850114:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1393 00000100:00100000:3.0:1789302867.854084:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.854166:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350491520 00000100:00100000:1.0:1789302867.854176:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350491520 00002000:00100000:1.0:1789302867.854226:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1c:0x0] (0->4095) req@ffff960f1db46a00 x1876219350491520/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:443/0 lens 488/0 e 0 to 0 dl 1789302878 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302867.854237:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1db46a00 with x1876219350491520 ext(0->4095) 00002000:00010000:1.0:1789302867.854244:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1db46a00 00000100:00100000:1.0:1789302867.854282:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 122 00000100:00100000:1.0:1789302867.854291:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350491520:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302867.854537:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350491520:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 246us (463us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302867.854546:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 122 00002000:00100000:1.0:1789302867.854552:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1c:0x0] (0->4095) req@ffff960f1db46a00 x1876219350491520/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:443/0 lens 488/4536 e 0 to 0 dl 1789302878 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302867.854561:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f1db46a00 with x1876219350491520 ext(0->4095) 00002000:00010000:1.0:1789302867.854565:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f1db46a00 00000100:00100000:3.0:1789302867.862180:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.862302:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350491648 00000100:00100000:1.0:1789302867.862315:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350491648 00000100:00100000:1.0:1789302867.862376:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1394 00000100:00100000:1.0:1789302867.862387:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32132:x1876219350491648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302867.862410:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302867.862433:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f380 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d3d expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.862468:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.862482:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.862494:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.862667:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184f380 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 55 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.862685:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 54 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.862694:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:1.0:1789302867.862698:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302867.862701:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8800, rc 0) 00000100:00100000:1.0:1789302867.862784:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32132:x1876219350491648:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 399us (613us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.862792:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1394 00000100:00100000:3.0:1789302867.878099:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.878208:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350491776 00000100:00100000:1.0:1789302867.878221:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350491776 00000100:00100000:1.0:1789302867.878273:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1395 00000100:00100000:1.0:1789302867.878283:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32132:x1876219350491776:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302867.878316:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302867.878424:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32132:x1876219350491776:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 142us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.878432:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1395 00000100:00100000:3.0:1789302867.893957:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302867.894322:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350491904 00000100:00100000:1.0:1789302867.894333:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350491904 00000100:00100000:1.0:1789302867.894390:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 396 00000100:00100000:1.0:1789302867.894401:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350491904:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302867.894457:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894474:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302867.894478:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302867.894484:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894521:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894553:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894560:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894574:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894580:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302867.894595:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894603:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894617:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302867.894624:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f38e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302867.894712:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350491904:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 312us (762us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.894720:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 396 00000100:00100000:3.0:1789302867.911485:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302867.911822:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350492160 00000100:00100000:0.0:1789302867.911833:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350492160 00000100:00100000:0.0:1789302867.911892:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1396 00000100:00100000:0.0:1789302867.911903:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32132:x1876219350492160:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302867.912741:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.912767:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 54 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.912776:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.912814:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 54 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.912830:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 54 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302867.912845:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371167872:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302867.913066:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded46c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302867.920050:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302867.921365:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350492288 00000100:00100000:2.0:1789302867.921390:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350492288 00010000:00010000:2.0:1789302867.921435:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 55 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302867.921477:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 417 00000100:00100000:2.0:1789302867.921486:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120b8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2348:x1876219350492288:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302867.921525:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302867.921533:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 55 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.921547:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 55 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.921568:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:2.0:1789302867.921574:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.921592:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.921601:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302867.921620:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302867.921675:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120b8c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2348:x1876219350492288:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 189us (1635us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302867.921682:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 417 00010000:00010000:0.0:1789302867.923115:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f387 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d3d expref: 54 pid: 5843 timeout: 804 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302867.923135:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70ed80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371167872:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302867.923160:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923170:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923176:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923379:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923387:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923394:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923400:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302867.923405:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923412:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923424:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:0.0:1789302867.923427:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302867.923431:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302867.923436:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f395 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302867.924051:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32132:x1876219350492160:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 12147us (12573us total) trans 4294968223 rc 0/0 00000100:00100000:0.0:1789302867.924061:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1396 00000040:00080000:3.0:1789302867.924491:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302867.924497:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302867.925036:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:3.0:1789302867.925045:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302867.938915:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302867.939024:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350492544 00000100:00100000:0.0:1789302867.939035:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350492544 00000100:00100000:0.0:1789302867.939109:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00100000:0.0:1789302867.939121:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32132:x1876219350492544:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302867.939441:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32132:x1876219350492544:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 322us (537us total) trans 4294968224 rc 0/0 00000100:00100000:0.0:1789302867.939449:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 281 00000100:00100000:0.0:1789302867.982330:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371168128:0@lo:400:kworker.0 00000100:00100000:0.0:1789302867.982400:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302867.982466:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371168640:0@lo:400:kworker.0 00000100:00100000:0.0:1789302867.982492:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302867.982538:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1789302867.982545:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371168000:0@lo:400:kworker.0 00000100:00100000:0.0:1789302867.982594:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302867.982622:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371168512:0@lo:400:kworker.0 00000100:00100000:0.0:1789302867.982648:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302867.982686:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371168000 00000100:00100000:0.0:1789302867.982694:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371168000 00000100:00100000:0.0:1789302867.982884:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 613 00000100:00100000:0.0:1789302867.982894:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371168000:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302867.982942:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371168000:12345-0@lo:400:kworker.0 Request processed in 50us (347us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302867.982949:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 613 00000100:00100000:0.0:1789302867.982994:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371168000:0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983075:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371168256:0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983105:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302867.983135:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302867.983141:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371168384:0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983162:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302867.983193:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371168128 00000100:00100000:1.0:1789302867.983197:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371168128 00000100:00100000:1.0:1789302867.983222:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371168512 00000100:00100000:1.0:1789302867.983224:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371168512 00000100:00100000:1.0:1789302867.983248:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371168256 00000100:00100000:1.0:1789302867.983250:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371168256 00000100:00100000:1.0:1789302867.983269:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1789302867.983275:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371168128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983325:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371168128:12345-0@lo:400:kworker.0 Request processed in 49us (924us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.983332:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:1.0:1789302867.983344:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:1.0:1789302867.983351:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371168512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983379:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371168512:12345-0@lo:400:kworker.0 Request processed in 29us (731us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.983384:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:1.0:1789302867.983395:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:1.0:1789302867.983400:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371168256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983432:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371168256:12345-0@lo:400:kworker.0 Request processed in 31us (325us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.983438:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:3.0:1789302867.983439:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371168640 00000100:00100000:3.0:1789302867.983445:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371168640 00000100:00100000:1.0:1789302867.983460:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371168384 00000100:00100000:1.0:1789302867.983466:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371168384 00000100:00100000:1.0:1789302867.983497:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:1.0:1789302867.983530:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371168384:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302867.983550:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1789302867.983561:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371168640:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983590:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371168384:12345-0@lo:400:kworker.0 Request processed in 42us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302867.983597:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1789302867.983618:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371168640:12345-0@lo:400:kworker.0 Request processed in 58us (1122us total) trans 0 rc 0/0 00000100:00080000:1.0:1789302867.983618:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302867.983624:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302867.983628:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00080000:1.0:1789302867.983628:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302867.983662:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371168384:0@lo:400:kworker.0 00000100:00100000:1.0:1789302867.983674:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371168256:0@lo:400:kworker.0 00000100:00100000:2.0:1789302867.983946:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371168512:0@lo:400:kworker.0 00000100:00100000:2.0:1789302867.983961:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371168128:0@lo:400:kworker.0 00000100:00100000:2.0:1789302867.983972:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371168640:0@lo:400:kworker.0 00000100:00100000:3.0:1789302868.507444:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371168768:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302868.507550:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302868.507632:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371168768 00000100:00100000:3.0:1789302868.507640:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371168768 00000100:00100000:3.0:1789302868.507698:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1789302868.507708:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371168768:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302868.510019:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3856662528 avail 3853516800 00000100:00100000:3.0:1789302868.510114:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371168768:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2405us (2561us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302868.510124:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:1.0:1789302868.511432:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371168768:0@lo:13:osp-pre-1-0.0 00000040:00080000:2.0:1789302868.564132:0:22127:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302868.564143:0:22127:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302868.564197:0:22127:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 42 count 4 00000040:00080000:2.0:1789302868.564202:0:22127:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000001:00080000:1.0:1789302868.574336:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968224 is committed 00000001:00080000:1.0:1789302868.574461:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968219 is committed 00000020:00080000:1.0:1789302868.574472:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 32 batchid updated 00000020:00080000:2.0:1789302868.574522:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 32 00000020:00080000:2.0:1789302868.574533:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 32 00010000:00010000:1.0:1789302868.630078:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302868.646335:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302868.647444:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350492672 00000100:00100000:3.0:1789302868.647456:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350492672 00000100:00100000:3.0:1789302868.647541:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 397 00000100:00100000:3.0:1789302868.647552:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350492672:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302868.647569:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302868.647598:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d44 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.647624:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302868.647629:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302868.647638:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.647657:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.647670:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302868.647739:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350492672:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 189us (1411us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302868.647746:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 397 00000100:00100000:1.0:1789302868.671220:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302868.671768:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350492928 00000100:00100000:0.0:1789302868.671780:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350492928 00000100:00100000:0.0:1789302868.671830:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1397 00000100:00100000:0.0:1789302868.671840:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32133:x1876219350492928:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302868.671859:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302868.671880:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f3a3 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d4b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.671995:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f3aa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.672012:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f3aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.672023:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f3aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302868.672036:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:0.0:1789302868.672214:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f3a3 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d4b expref: 108 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.672243:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f3aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d4b expref: 107 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.672252:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:0.0:1789302868.672255:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:0.0:1789302868.672258:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b600, rc 0) 00000100:00100000:0.0:1789302868.672316:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32133:x1876219350492928:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 478us (1108us total) trans 4294968225 rc 0/0 00000100:00100000:0.0:1789302868.672324:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1397 00000100:00100000:0.0:1789302868.681160:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302868.681683:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350493056 00000100:00100000:3.0:1789302868.681693:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350493056 00000100:00100000:3.0:1789302868.681761:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 398 00000100:00100000:3.0:1789302868.681774:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2345:x1876219350493056:12345-192.168.203.11@tcp:1:lfs.0 00010000:00010000:3.0:1789302868.681821:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.681840:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 9 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.681850:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.681858:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302868.681862:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302868.681867:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.681905:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.681920:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 18 pid: 6695 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302868.682155:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371168896:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302868.682268:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 9 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.682279:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 9 pid: 8571 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302868.682289:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371169024:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302868.682315:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cde4e40 going to sleep for 11 seconds 00000100:00100000:1.0:1789302868.689490:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371168896:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302868.689610:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371169024:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302868.689645:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302868.699467:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302868.699586:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302868.699828:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350493184 00000100:00100000:3.0:1789302868.699838:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350493184 00010000:00010000:3.0:1789302868.699880:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 19 pid: 6695 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302868.699916:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350493312 00000100:00100000:3.0:1789302868.699920:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350493312 00010000:00010000:3.0:1789302868.699945:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 10 pid: 8571 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302868.699975:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 418 00000100:00100000:3.0:1789302868.699986:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2347:x1876219350493184:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302868.700000:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302868.700023:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 19 pid: 6695 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700036:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 19 pid: 6695 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700057:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d356e686/0x64fe43095184f39c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d44 expref: 19 pid: 6695 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700071:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302868.700124:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350493184:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 140us (664us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302868.700131:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 418 00000100:00100000:3.0:1789302868.700148:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 419 00000100:00100000:3.0:1789302868.700155:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350493312:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302868.700164:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302868.700172:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 10 pid: 8571 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700182:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 10 pid: 8571 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700196:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f379 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d36 expref: 10 pid: 8571 timeout: 805 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700207:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302868.700212:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302868.700218:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700227:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.700398:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302868.700420:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350493312:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 264us (843us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302868.700426:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 419 00010000:00010000:0.0:1789302868.703772:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703792:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703828:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703837:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703847:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703854:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302868.703859:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703867:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703886:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302868.703893:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe43095184f3b8 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302868.704000:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350493056:12345-192.168.203.11@tcp:1:lfs.0 Request processed in 22224us (22845us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302868.704009:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 398 00000100:00100000:1.0:1789302868.710126:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302868.710829:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350493568 00000100:00100000:2.0:1789302868.710840:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350493568 00000100:00100000:2.0:1789302868.710896:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 282 00000100:00100000:2.0:1789302868.710907:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ea850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32133:x1876219350493568:12345-192.168.203.11@tcp:35:lfs.0 00000004:00010000:2.0:1789302868.710979:0:12976:0:(mdt_open.c:1998:mdt_hsm_release()) ### [0x200000402:0x2a:0x0] lease broken? 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f3aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d4b expref: 107 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302868.711226:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968226 00000001:00080000:2.0:1789302868.712464:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968227 00010000:00010000:2.0:1789302868.712859:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.712881:0:12976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.712891:0:12976:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 110 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.712900:0:12976:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 5/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.713017:0:12976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 110 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.713040:0:12976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 110 pid: 5845 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302868.713057:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371169152:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302868.713199:0:12976:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 49 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.713209:0:12976:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 49 pid: 5844 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302868.713218:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cb800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371169280:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302868.713250:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03529600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302868.718145:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302868.718181:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302868.718213:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350493696 00000100:00100000:1.0:1789302868.718223:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350493696 00010000:00010000:1.0:1789302868.718260:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 111 pid: 5845 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302868.718290:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350493824 00000100:00100000:1.0:1789302868.718293:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350493824 00010000:00010000:1.0:1789302868.718306:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 50 pid: 5844 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302868.718327:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 420 00000100:00100000:1.0:1789302868.718336:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:2347:x1876219350493696:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302868.718350:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302868.718357:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 111 pid: 5845 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718370:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 111 pid: 5845 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718393:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:1.0:1789302868.718398:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718407:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718414:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302868.718463:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:2347:x1876219350493696:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 128us (326us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302868.718474:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 420 00000100:00100000:1.0:1789302868.718494:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 421 00000100:00100000:1.0:1789302868.718524:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2348:x1876219350493824:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302868.718534:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302868.718539:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 50 pid: 5844 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718548:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 50 pid: 5844 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718559:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:1.0:1789302868.718562:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718570:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718587:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302868.718606:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302868.718627:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2348:x1876219350493824:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 126us (444us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302868.718634:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 421 00010000:00010000:2.0:1789302868.722532:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f33a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d13 expref: 110 pid: 5845 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302868.722557:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371169152:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302868.722571:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f309 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cfe expref: 49 pid: 5844 timeout: 805 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302868.722593:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cb800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000_UUID:12976:1876219371169280:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302868.722619:0:12976:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.722629:0:12976:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302868.722636:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302868.722972:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968228 00000001:00080000:2.0:1789302868.723188:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968229 00000001:00080000:2.0:1789302868.723243:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968230 00000100:00100000:2.0:1789302868.723333:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cb800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371169408:0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:2.0:1789302868.723399:0:12976:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302868.723443:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdac00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302868.723608:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371169408 00000100:00100000:0.0:1789302868.723616:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371169408 00000100:00100000:0.0:1789302868.723675:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 69 00000100:00100000:0.0:1789302868.723686:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371169408:12345-0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:0.0:1789302868.723783:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371169408:12345-0@lo:1000:mdt_rdpg00_002.0 Request processed in 99us (383us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302868.723790:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 69 00000100:00100000:3.0:1789302868.724118:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cb800 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371169408:0@lo:1000:mdt_rdpg00_002.0 00000001:00080000:3.0:1789302868.724493:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968231 00000100:00100000:3.0:1789302868.724601:0:12976:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371169536:0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:3.0:1789302868.724642:0:12976:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302868.724674:0:12976:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03529e40 going to sleep for 11 seconds 00000100:00100000:0.0:1789302868.725183:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371169536 00000100:00100000:0.0:1789302868.725191:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371169536 00000100:00100000:0.0:1789302868.725237:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 70 00000100:00100000:0.0:1789302868.725247:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:12976:x1876219371169536:12345-0@lo:1000:mdt_rdpg00_002.0 00000100:00100000:0.0:1789302868.725530:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:12976:x1876219371169536:12345-0@lo:1000:mdt_rdpg00_002.0 Request processed in 283us (886us total) trans 4294967378 rc 0/0 00000100:00100000:0.0:1789302868.725537:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 70 00000100:00100000:3.0:1789302868.726125:0:12976:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job mdt_rdpg00_002:lustre-MDT0000-mdtlov_UUID:12976:1876219371169536:0@lo:1000:mdt_rdpg00_002.0 00010000:00010000:3.0:1789302868.726152:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.726176:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.726186:0:12976:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.726192:0:12976:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbfc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302868.726197:0:12976:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.726205:0:12976:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.726218:0:12976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:3.0:1789302868.726221:0:12976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 28 00010000:00010000:3.0:1789302868.726226:0:12976:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302868.726232:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3bf lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x28/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302868.726444:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000401:0x50:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000040:00080000:3.0:1789302868.726623:0:12976:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].14, 64 off9160 00000004:00080000:3.0:1789302868.726636:0:12976:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 705737133394ns, next at 706737130429ns, rc = 0 00000001:00080000:3.0:1789302868.726646:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968232 00010000:00010000:3.0:1789302868.727642:0:12976:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3c500) 00010000:00010000:3.0:1789302868.727765:0:12976:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 4 00010000:00010000:3.0:1789302868.727776:0:12976:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f3aa lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x4/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d4b expref: 114 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302868.731808:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ea850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:32133:x1876219350493568:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 20893us (21682us total) trans 4294968233 rc 0/0 00000100:00100000:3.0:1789302868.731828:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 282 00000020:00080000:3.0:1789302868.731837:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 8 transactions per RPC 00000100:00100000:2.0:1789302869.004791:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302869.004816:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371169664:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302869.004883:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302869.004997:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371169664 00000100:00100000:3.0:1789302869.005005:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371169664 00000100:00100000:3.0:1789302869.005072:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 211 00000100:00100000:3.0:1789302869.005082:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371169664:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302869.005116:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302869.005167:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371169664:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (283us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302869.005175:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 211 00000100:00100000:3.0:1789302869.005217:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371169664:0@lo:13:osp-pre-0-0.0 00000100:00080000:1.0:1789302869.005541:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302869.266303:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302869.266346:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302869.266369:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302869.266395:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350494208 00000100:00100000:3.0:1789302869.266404:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350494208 00000100:00100000:3.0:1789302869.266471:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350494080 00000100:00100000:3.0:1789302869.266475:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350494080 00000100:00100000:3.0:1789302869.266499:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00100000:3.0:1789302869.266530:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350494208:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302869.266618:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350494208:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 89us (271us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302869.266627:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00100000:3.0:1789302869.266655:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 511 00000100:00100000:3.0:1789302869.266663:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2347:x1876219350494080:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302869.266694:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2347:x1876219350494080:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (323us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302869.266700:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 511 00000100:00100000:0.0:1789302869.269624:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350493952 00000100:00100000:0.0:1789302869.269639:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350493952 00000100:00100000:0.0:1789302869.269831:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 614 00000100:00100000:0.0:1789302869.269851:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350493952:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:1.0:1789302869.270093:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302869.270129:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350493952:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 288us (3834us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302869.270137:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 614 00010000:00010000:1.0:1789302869.644728:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302869.738890:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302869.738929:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350494464 00000100:00100000:2.0:1789302869.738940:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350494464 00000100:00100000:2.0:1789302869.739011:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1398 00000100:00100000:2.0:1789302869.739022:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:32142:x1876219350494464:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302869.739045:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302869.739065:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3c6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d52 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739143:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739157:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739167:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739192:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739199:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739206:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739494:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f3db lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739522:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f3db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739530:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f3db lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302869.739540:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:2.0:1789302869.739636:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739644:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739651:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739657:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302869.739662:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739674:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739685:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:2.0:1789302869.739688:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302869.739693:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739699:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f3cd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739711:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739717:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739723:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739727:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302869.739729:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739735:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739742:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xb6c33e9e (ffff960f10d3c200) 00010000:00010000:2.0:1789302869.739744:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302869.739747:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739757:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f3d4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0xb6c33e9e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739802:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbfc00/0x64fe43095184f3c6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d52 expref: 116 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739816:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f3db lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d52 expref: 115 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.739824:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302869.739826:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302869.739829:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302869.739877:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:32142:x1876219350494464:12345-192.168.203.11@tcp:101:cat.0 Request processed in 858us (998us total) trans 4294968234 rc 0/0 00000100:00100000:2.0:1789302869.739884:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1398 00000100:00080000:3.0:1789302869.740232:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302869.758278:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302869.758563:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350494720 00000100:00100000:2.0:1789302869.758588:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350494720 00000100:00100000:2.0:1789302869.758640:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1399 00000100:00100000:2.0:1789302869.758650:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:32143:x1876219350494720:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302869.758675:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302869.758697:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f3e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d59 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.758766:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.758780:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.758791:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302869.758830:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00000004:00200000:2.0:1789302869.758833:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302869.758852:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.758861:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.758885:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f3e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 117 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.758898:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 116 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302869.758906:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302869.758909:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302869.758913:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:2.0:1789302869.758969:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:32143:x1876219350494720:12345-192.168.203.11@tcp:101:cat.0 Request processed in 320us (700us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302869.758976:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1399 00000100:00100000:1.0:1789302869.761932:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302869.762722:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350494848 00000100:00100000:3.0:1789302869.762735:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350494848 00000100:00100000:3.0:1789302869.762809:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 65 00000100:00100000:3.0:1789302869.762820:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:32142:x1876219350494848:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:3.0:1789302869.762836:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302869.762861:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f3db lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d52 expref: 116 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302869.762881:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:3.0:1789302869.762887:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302869.762942:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:32142:x1876219350494848:12345-192.168.203.11@tcp:103:cat.0 Request processed in 124us (1019us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302869.762949:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 65 00000100:00100000:1.0:1789302869.764883:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302869.765346:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350494976 00000100:00100000:2.0:1789302869.765356:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350494976 00000100:00100000:2.0:1789302869.765415:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1400 00000100:00100000:2.0:1789302869.765425:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:32142:x1876219350494976:12345-192.168.203.11@tcp:58:cat.0 00000004:00020000:2.0:1789302869.765531:0:5843:0:(mdt_hsm_cdt_client.c:349:mdt_hsm_register_hal()) cfs_race id 18b sleeping 00010000:00010000:2.0:1789302870.669473:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302870.988792:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302870.988801:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302870.989114:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 42 count 2 00000040:00080000:1.0:1789302870.989121:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302871.693484:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302872.716712:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302873.100852:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371169920:0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.100935:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302873.100945:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371169792:0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.100979:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302873.100999:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371170432:0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101002:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371170304:0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101023:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302873.101023:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302873.101048:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371170048:0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.101065:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371169920 00000100:00100000:3.0:1789302873.101069:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302873.101073:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371169920 00000100:00100000:3.0:1789302873.101085:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1789302873.101091:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371170176:0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.101109:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371170304 00000100:00100000:2.0:1789302873.101112:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371170304 00000100:00100000:3.0:1789302873.101122:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302873.101147:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371170048 00000100:00100000:3.0:1789302873.101148:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371169792 00000100:00100000:3.0:1789302873.101151:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371169792 00000100:00100000:2.0:1789302873.101151:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371170048 00000100:00100000:2.0:1789302873.101171:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:3.0:1789302873.101173:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:3.0:1789302873.101179:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371169792:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.101179:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371169920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101222:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371169792:12345-0@lo:400:kworker.0 Request processed in 43us (242us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302873.101222:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371169920:12345-0@lo:400:kworker.0 Request processed in 45us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302873.101230:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:2.0:1789302873.101230:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:2.0:1789302873.101245:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:2.0:1789302873.101250:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371170304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101258:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:3.0:1789302873.101264:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371170048:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.101274:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371170304:12345-0@lo:400:kworker.0 Request processed in 25us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302873.101279:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:3.0:1789302873.101291:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371170048:12345-0@lo:400:kworker.0 Request processed in 28us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302873.101295:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371170432 00000100:00100000:3.0:1789302873.101297:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1789302873.101298:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371170432 00000100:00080000:3.0:1789302873.101313:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302873.101317:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302873.101320:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371170176 00000100:00100000:2.0:1789302873.101322:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371170176 00000100:00100000:3.0:1789302873.101328:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:3.0:1789302873.101334:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371170432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.101340:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1789302873.101345:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371170176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101362:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371170432:12345-0@lo:400:kworker.0 Request processed in 28us (339us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302873.101367:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:2.0:1789302873.101368:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371170176:12345-0@lo:400:kworker.0 Request processed in 24us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302873.101373:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00080000:3.0:1789302873.101383:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302873.101399:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371170176:0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.101399:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371170432:0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101408:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c91500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371170048:0@lo:400:kworker.0 00000100:00100000:2.0:1789302873.101408:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371169920:0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101431:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371169792:0@lo:400:kworker.0 00000100:00100000:3.0:1789302873.101442:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cbb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371170304:0@lo:400:kworker.0 00000001:00080000:1.0:1789302873.159382:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967378 is committed 00000004:00080000:3.0:1789302873.612754:0:8001:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 710623249148ns, next at 711623246879ns, rc = 0 00000100:00100000:2.0:1789302873.612801:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302873.612822:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371170560:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302873.612887:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302873.612996:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371170560 00000100:00100000:3.0:1789302873.613005:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371170560 00000100:00100000:3.0:1789302873.613071:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:3.0:1789302873.613081:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371170560:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302873.613115:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302873.613182:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371170560:12345-0@lo:13:osp-pre-1-0.0 Request processed in 101us (294us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302873.613189:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1789302873.613330:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967378 gen 2 00000004:00080000:1.0:1789302873.613336:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967378 committed transno 4294967378 00000100:00100000:1.0:1789302873.613361:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371170560:0@lo:13:osp-pre-1-0.0 00000001:00080000:3.0:1789302873.693946:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968225 is committed 00000001:00080000:3.0:1789302873.694238:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968234 is committed 00000004:00080000:3.0:1789302873.694259:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 17 committed 00000004:00080000:3.0:1789302873.694277:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 16 committed 00000020:00080000:3.0:1789302873.694286:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f098a2180 refcount 2 committed 0 result 0 batchid 33 00000020:00080000:3.0:1789302873.694291:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09762000 obd lustre-OST0001-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f0cde4e40 00000020:00080000:3.0:1789302873.694294:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f09762500 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3f753c00 00000100:00100000:0.0:1789302873.694531:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789302873.694560:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371170688:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789302873.694627:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302873.694784:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371170688 00000100:00100000:2.0:1789302873.694793:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371170688 00000100:00100000:2.0:1789302873.694860:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1789302873.694871:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371170688:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1789302873.694897:0:6695:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:28 count 1 00002000:00010000:2.0:1789302873.694930:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1c:0x0] initial LVB size: 158, mtime: 0x6aa69853, atime: 0x0, ctime: 0x6aa69853, blocks: 0xb 00010000:00010000:2.0:1789302873.694938:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.694953:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=710s v=(0 0) 00010000:00010000:2.0:1789302873.694957:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1c:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302873.694963:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.694980:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.694989:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.695003:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.695022:0:6695:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789302873.695611:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371170688:12345-0@lo:6:osp-syn-1-0.0 Request processed in 740us (981us total) trans 4294967379 rc 0/0 00000100:00100000:2.0:1789302873.695619:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00010000:00010000:2.0:1789302873.695646:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302873.695652:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.695660:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302873.695664:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.695675:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.695691:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.695699:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe43095184f3f7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302873.695717:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302873.695719:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302873.695721:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1789302873.696203:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f70e300/2, rc 0, transno 4294967379 00000100:00100000:0.0:1789302873.696212:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371170688:0@lo:6:osp-syn-1-0.0 00000020:00080000:1.0:1789302873.696333:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 33 committed batchid 32 00000020:00080000:1.0:1789302873.696482:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 32 00010000:00010000:1.0:1789302873.740792:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302874.126429:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302874.126451:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f708a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371170816:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302874.126534:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302874.126630:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371170816 00000100:00100000:1.0:1789302874.126639:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371170816 00000100:00100000:1.0:1789302874.126683:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789302874.126692:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371170816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302874.126730:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 86704128 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302874.126778:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371170816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (244us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302874.126784:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 213 00000100:00100000:1.0:1789302874.126829:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f708a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371170816:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302874.380786:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302874.380838:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302874.381203:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350495232 00000100:00100000:0.0:1789302874.381212:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350495232 00000100:00100000:0.0:1789302874.381286:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 495 00000100:00100000:0.0:1789302874.381296:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350495232:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302874.381364:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350495232:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 70us (587us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302874.381370:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 495 00000100:00100000:3.0:1789302874.381751:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350495104 00000100:00100000:3.0:1789302874.381756:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350495104 00000100:00100000:3.0:1789302874.381793:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 616 00000100:00100000:3.0:1789302874.381800:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350495104:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302874.381839:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350495104:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 39us (1000us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302874.381845:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 616 00000100:00080000:2.0:1789302874.382290:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302874.385476:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302874.385563:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302874.385600:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302874.385646:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302874.385667:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302874.385714:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350495488 00000100:00100000:2.0:1789302874.385723:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350495488 00000100:00100000:2.0:1789302874.385775:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350495744 00000100:00100000:2.0:1789302874.385778:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350495744 00000100:00100000:2.0:1789302874.385797:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350495360 00000100:00100000:2.0:1789302874.385800:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350495360 00000100:00100000:2.0:1789302874.385814:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350495872 00000100:00100000:2.0:1789302874.385816:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350495872 00000100:00100000:2.0:1789302874.385830:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 514 00000100:00100000:2.0:1789302874.385839:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350495488:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302874.385908:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350495488:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302874.385915:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 514 00000100:00100000:2.0:1789302874.385932:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 515 00000100:00100000:2.0:1789302874.385938:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350495744:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302874.385961:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350495744:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302874.385967:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 515 00000100:00100000:2.0:1789302874.385979:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 516 00000100:00100000:2.0:1789302874.385985:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350495360:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302874.386007:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:2348:x1876219350495360:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (407us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302874.386012:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 516 00000100:00100000:2.0:1789302874.386023:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 517 00000100:00100000:2.0:1789302874.386028:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350495872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302874.386067:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350495872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 39us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302874.386074:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 517 00000100:00080000:2.0:1789302874.386095:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302874.386099:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302874.387475:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350495616 00000100:00100000:3.0:1789302874.387486:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350495616 00000100:00100000:3.0:1789302874.387595:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 496 00000100:00100000:3.0:1789302874.387616:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350495616:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302874.387974:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350495616:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 369us (2325us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302874.387982:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 496 00000100:00080000:2.0:1789302874.391902:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302874.766218:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00020000:2.0:1789302875.148659:0:5843:0:(mdt_hsm_cdt_client.c:349:mdt_hsm_register_hal()) cfs_fail_race id 18b awake: rc=0 00010000:00010000:2.0:1789302875.156906:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.156932:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.156942:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.156986:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.157004:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 105 pid: 18427 timeout: 812 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302875.157027:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371170944:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302875.157338:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdac00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302875.167943:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302875.168932:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350496000 00000100:00100000:3.0:1789302875.168942:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350496000 00010000:00010000:3.0:1789302875.168984:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 106 pid: 18427 timeout: 812 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302875.169079:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 422 00000100:00100000:3.0:1789302875.169090:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350496000:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302875.169106:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302875.169113:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 106 pid: 18427 timeout: 812 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302875.169126:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 106 pid: 18427 timeout: 812 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302875.169148:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:3.0:1789302875.169153:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302875.169163:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302875.169170:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302875.169188:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302875.169922:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350496000:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 829us (1981us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302875.169931:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 422 00010000:00010000:1.0:1789302875.171069:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f3f0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d59 expref: 105 pid: 18427 timeout: 812 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302875.171093:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d9c00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371170944:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302875.171118:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302875.171128:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302875.171134:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302875.171472:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].42, 136 off13904 00000100:00100000:1.0:1789302875.171636:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32142:x1876219350494976:12345-192.168.203.11@tcp:58:cat.0 Request processed in 5406211us (5406759us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302875.171647:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1400 00000100:00100000:0.0:1789302875.181147:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302875.181527:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350496128 00000100:00100000:2.0:1789302875.181546:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350496128 00000100:00100000:2.0:1789302875.181615:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1401 00000100:00100000:2.0:1789302875.181626:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32142:x1876219350496128:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302875.181646:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302875.181667:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d60 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.181730:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 2/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.181742:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 4/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.181775:0:18427:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.181789:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.181802:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302875.181806:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302875.181809:0:18427:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 3/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302875.181817:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:2.0:1789302875.181878:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32142:x1876219350496128:12345-192.168.203.11@tcp:101:cat.0 Request processed in 253us (740us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302875.181885:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1401 00010000:00010000:2.0:1789302875.792712:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302876.109208:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302876.109219:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302876.109258:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 43 count 3 00000040:00080000:1.0:1789302876.109265:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302876.109392:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f708a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371171072:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302876.109547:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ded4240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302876.118052:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302876.118225:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350496256 00000100:00100000:2.0:1789302876.118235:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350496256 00000100:00100000:2.0:1789302876.118328:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1402 00000100:00100000:2.0:1789302876.118339:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32145:x1876219350496256:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302876.118376:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302876.118485:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32145:x1876219350496256:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 147us (453us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302876.118492:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1402 00000100:00100000:1.0:1789302876.118836:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f708a80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371171072:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00080000:3.0:1789302876.119249:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302876.133866:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.133917:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350496384 00000100:00100000:1.0:1789302876.133928:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350496384 00000100:00100000:1.0:1789302876.133989:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1403 00000100:00100000:1.0:1789302876.134001:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32145:x1876219350496384:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:1.0:1789302876.134074:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134091:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134103:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134118:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134125:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134131:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134155:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134163:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134168:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302876.134870:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00000004:00200000:1.0:1789302876.134876:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302876.134892:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134909:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134916:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134924:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134929:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302876.134933:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134940:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134952:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:1.0:1789302876.134954:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302876.134958:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134964:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f40c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134979:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134984:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134989:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.134994:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302876.134996:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135001:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135008:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x98b61b69 (ffff960f2c94da00) 00010000:00010000:1.0:1789302876.135010:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302876.135012:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135016:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f413 lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x98b61b69 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135031:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135038:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135044:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135049:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302876.135051:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135057:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135068:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:1.0:1789302876.135071:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302876.135073:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.135079:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f41a lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.135148:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038da300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32145:x1876219350496384:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 1148us (1291us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302876.135156:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1403 00000100:00100000:0.0:1789302876.155074:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.155875:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350496640 00000100:00100000:1.0:1789302876.155886:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350496640 00000100:00100000:1.0:1789302876.155954:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1404 00000100:00100000:1.0:1789302876.155965:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32145:x1876219350496640:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302876.155986:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302876.156009:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f421 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d67 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156108:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156123:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156136:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302876.156146:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2a:0x0]: returning size 158 00010000:00010000:1.0:1789302876.156283:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156292:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156299:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156306:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302876.156310:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156317:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156330:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:1.0:1789302876.156333:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302876.156338:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156343:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f428 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2a:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156383:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f421 lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d67 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156393:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:1.0:1789302876.156395:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302876.156399:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f421 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d67 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.156414:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9600, rc 0) 00000100:00100000:1.0:1789302876.156491:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32145:x1876219350496640:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 527us (1427us total) trans 4294968235 rc 301/301 00000100:00100000:1.0:1789302876.156498:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1404 00000100:00100000:0.0:1789302876.171129:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.176995:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350496768 00000100:00100000:3.0:1789302876.177011:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350496768 00000100:00100000:3.0:1789302876.177448:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1405 00000100:00100000:3.0:1789302876.177461:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32145:x1876219350496768:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302876.177499:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2a:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302876.177659:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32145:x1876219350496768:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 200us (6538us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.177677:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1405 00000100:00100000:3.0:1789302876.186297:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.186486:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350497024 00000100:00100000:1.0:1789302876.186496:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350497024 00000100:00100000:1.0:1789302876.186944:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1406 00000100:00100000:1.0:1789302876.186956:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32145:x1876219350497024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302876.186978:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302876.187001:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f436 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d6e expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.188255:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f43d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.188272:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f43d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.188285:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f43d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302876.188297:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2b:0x0]: returning size 0 00010000:00010000:1.0:1789302876.188352:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f436 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d6e expref: 53 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.188371:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f43d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d6e expref: 52 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.188384:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f2c94da00) 00010000:00010000:1.0:1789302876.188387:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302876.188390:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9a00, rc 0) 00000100:00100000:1.0:1789302876.188450:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32145:x1876219350497024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1497us (2162us total) trans 4294968236 rc 0/0 00000100:00100000:1.0:1789302876.188457:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1406 00000100:00100000:0.0:1789302876.196644:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302876.200106:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350497280 00000100:00100000:2.0:1789302876.200124:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350497280 00000100:00100000:2.0:1789302876.200210:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1407 00000100:00100000:2.0:1789302876.200222:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32145:x1876219350497280:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302876.200285:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302876.200311:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f43d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d6e expref: 52 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.200342:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f2c94da00) 00010000:00010000:2.0:1789302876.200353:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302876.200377:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.200392:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 104 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.200401:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 4/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.200435:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 104 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.200449:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 104 pid: 5843 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302876.200462:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038d8380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371171200:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302876.200636:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdaf00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302876.211945:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302876.212060:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038d8380 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371171200:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302876.212090:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.212447:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350497408 00000100:00100000:3.0:1789302876.212456:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350497408 00010000:00010000:3.0:1789302876.212652:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 105 pid: 5843 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.212705:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 423 00000100:00100000:3.0:1789302876.212715:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd07450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:2348:x1876219350497408:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302876.212731:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302876.212738:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 105 pid: 5843 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212750:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 105 pid: 5843 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212774:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f34f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4f:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d21 expref: 105 pid: 5843 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212790:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:3.0:1789302876.212795:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212804:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212812:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212834:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302876.212882:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd07450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2348:x1876219350497408:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 169us (947us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.212889:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 423 00010000:00010000:3.0:1789302876.212958:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212966:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.212992:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.213002:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.213010:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214454:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214469:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214475:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214481:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302876.214485:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214492:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214535:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:3.0:1789302876.214539:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302876.214544:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214550:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184f44b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x4f:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214563:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214568:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214573:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214579:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302876.214581:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214586:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214594:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0xeec8aade (ffff960f0cdf2400) 00010000:00010000:3.0:1789302876.214596:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302876.214598:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.214616:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f452 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x4f:0x0].0xeec8aade bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.214715:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32145:x1876219350497280:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 14493us (18080us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302876.214724:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1407 00000100:00100000:0.0:1789302876.220051:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.220190:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350497536 00000100:00100000:3.0:1789302876.220199:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350497536 00000100:00100000:3.0:1789302876.220260:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1408 00000100:00100000:3.0:1789302876.220271:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32145:x1876219350497536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302876.220291:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302876.220314:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f459 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d75 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.220350:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.220369:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.220382:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.220484:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f459 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 53 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.220545:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.220555:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302876.220558:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302876.220561:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302876.220611:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32145:x1876219350497536:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 342us (582us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.220630:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1408 00000100:00080000:2.0:1789302876.226027:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302876.229475:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.229773:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350497792 00000100:00100000:1.0:1789302876.229783:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350497792 00000100:00100000:1.0:1789302876.229852:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1409 00000100:00100000:1.0:1789302876.229864:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32145:x1876219350497792:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302876.229946:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.229961:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.229972:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302876.230488:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dbcf7b2b reply state 0000000000daaa37 transno 4294968237 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.230497:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302876.230959:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32145:x1876219350497792:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1095us (1490us total) trans 4294968237 rc 0/0 00000100:00100000:1.0:1789302876.230966:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1409 00000100:00100000:0.0:1789302876.237421:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302876.237586:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.237604:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.237613:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.237623:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302876.237628:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.237635:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.237651:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:2.0:1789302876.237654:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1789302876.237659:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.237677:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f467 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.238322:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350497920 00000100:00100000:3.0:1789302876.238333:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350497920 00000100:00100000:3.0:1789302876.238427:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1410 00000100:00100000:3.0:1789302876.238437:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32145:x1876219350497920:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302876.238460:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302876.238477:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d7c expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.238576:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 54 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.238590:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 54 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.238607:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302876.238609:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302876.238612:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302876.238686:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32145:x1876219350497920:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 250us (1274us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.238693:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1410 00000100:00100000:0.0:1789302876.242934:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.243012:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350498048 00000100:00100000:3.0:1789302876.243023:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350498048 00000100:00100000:3.0:1789302876.243089:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:3.0:1789302876.243099:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32145:x1876219350498048:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 00000100:00100000:3.0:1789302876.243242:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32145:x1876219350498048:12345-192.168.203.11@tcp:49:lhsmtool_posix.0 Request processed in 144us (318us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.243250:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 283 00000100:00100000:0.0:1789302876.247689:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.247892:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350498304 00000100:00100000:3.0:1789302876.247902:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350498304 00000100:00100000:3.0:1789302876.247960:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:3.0:1789302876.247970:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32145:x1876219350498304:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302876.248235:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32145:x1876219350498304:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 265us (567us total) trans 4294968238 rc 0/0 00000100:00100000:3.0:1789302876.248244:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 284 00000100:00100000:0.0:1789302876.251767:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.251967:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350498560 00000100:00100000:3.0:1789302876.251978:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350498560 00000100:00100000:3.0:1789302876.252045:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1411 00000100:00100000:3.0:1789302876.252056:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32145:x1876219350498560:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302876.252166:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32145:x1876219350498560:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 111us (406us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.252173:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1411 00000100:00100000:2.0:1789302876.269266:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.269362:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350498816 00000100:00100000:1.0:1789302876.269374:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350498816 00000100:00100000:1.0:1789302876.269438:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1412 00000100:00100000:1.0:1789302876.269449:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32145:x1876219350498816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302876.269469:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302876.269491:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f475 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d83 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.269647:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.269663:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.269682:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.269717:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.269731:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 56 pid: 5844 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.269744:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f708a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371171328:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302876.269832:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e039ffd80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302876.286003:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.287722:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f708a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371171328:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302876.287758:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.300252:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350498944 00000100:00100000:1.0:1789302876.300268:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350498944 00010000:00010000:1.0:1789302876.300920:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 57 pid: 5844 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.300981:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 424 00000100:00100000:1.0:1789302876.300993:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:2347:x1876219350498944:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302876.301019:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302876.301027:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 57 pid: 5844 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.301039:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 57 pid: 5844 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.301059:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f46e lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d7c expref: 57 pid: 5844 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.301076:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302876.301082:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.301092:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.301098:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.301134:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302876.301200:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2347:x1876219350498944:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 210us (15208us total) trans 0 rc 0/0 00010000:00010000:3.0:1789302876.301206:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.301211:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 424 00010000:00010000:3.0:1789302876.301221:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302876.301242:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2b:0x0]: returning size 0 00000004:00080000:3.0:1789302876.301571:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1d:0x0], index 0: 0 00010000:00010000:3.0:1789302876.301993:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302009:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302016:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302022:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302876.302027:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302305:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302318:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302876.302322:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1789302876.302326:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302332:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f47c lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302392:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f475 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d83 expref: 56 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302403:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4f:0x0].0x0 (ffff960f2c94de00) 00010000:00010000:3.0:1789302876.302405:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302876.302409:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f475 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4f:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d83 expref: 56 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.302418:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc 0) 00000100:00100000:3.0:1789302876.302461:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32145:x1876219350498816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 33014us (33204us total) trans 4294968239 rc 301/301 00000100:00100000:3.0:1789302876.302469:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1412 00000100:00100000:2.0:1789302876.312832:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.313149:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350499200 00000100:00100000:3.0:1789302876.313160:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350499200 00000100:00100000:3.0:1789302876.313297:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 285 00000100:00100000:3.0:1789302876.313308:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32145:x1876219350499200:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302876.317462:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038daa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32145:x1876219350499200:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1719us (2202us total) trans 4294968240 rc 0/0 00000100:00100000:3.0:1789302876.317481:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 285 00000100:00100000:2.0:1789302876.321587:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.322270:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350499328 00000100:00100000:3.0:1789302876.322281:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350499328 00000100:00100000:3.0:1789302876.322351:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1413 00000100:00100000:3.0:1789302876.322362:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32145:x1876219350499328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302876.322386:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302876.322408:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f48a lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d8a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.322479:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f491 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.322493:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f491 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.322539:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f491 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302876.322576:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2b:0x0]: returning size 0 00000004:00200000:3.0:1789302876.322581:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302876.322607:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f491 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.322635:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f48a lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d8a expref: 58 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.322655:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f491 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d8a expref: 57 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.322664:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302876.322675:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1789302876.322678:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:3.0:1789302876.327406:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32145:x1876219350499328:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 5034us (5815us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.327420:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1413 00000100:00100000:2.0:1789302876.332751:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302876.332821:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350499456 00000100:00100000:0.0:1789302876.332832:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350499456 00000100:00100000:0.0:1789302876.332921:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 400 00000100:00100000:0.0:1789302876.332933:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350499456:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302876.332951:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302876.332973:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d91 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302876.333015:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302876.333035:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=713s v=(0 0) 00010000:00010000:0.0:1789302876.333040:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302876.333049:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.333067:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.333080:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345be00, rc 0) 00000100:00100000:0.0:1789302876.333142:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038d9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350499456:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 211us (400us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302876.333149:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 400 00000100:00080000:2.0:1789302876.333221:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302876.347009:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302876.347836:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350499712 00000100:00100000:0.0:1789302876.347849:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350499712 00000100:00100000:0.0:1789302876.347928:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1414 00000100:00100000:0.0:1789302876.347939:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32145:x1876219350499712:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302876.348111:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32145:x1876219350499712:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 173us (1113us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302876.348119:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1414 00000100:00100000:3.0:1789302876.359400:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302876.359562:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350499840 00000100:00100000:2.0:1789302876.359572:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350499840 00000100:00100000:2.0:1789302876.359701:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 401 00000100:00100000:2.0:1789302876.359714:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:32145:x1876219350499840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302876.359733:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302876.359751:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d98 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.359787:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.359797:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302876.359801:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302876.359808:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.359842:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.359854:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 11 pid: 6697 timeout: 813 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302876.359869:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371171456:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302876.359965:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdaf00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302876.363360:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371171456:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302876.363400:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.363421:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:3.0:1789302876.363483:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:32145:x1876219350499840:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 3772us (4091us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.363492:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 401 00000100:00100000:0.0:1789302876.371381:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.371495:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350499968 00000100:00100000:3.0:1789302876.371520:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350499968 00010000:00010000:3.0:1789302876.371575:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 11 pid: 6697 timeout: 813 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.371780:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 425 00000100:00100000:3.0:1789302876.371792:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350499968:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302876.371807:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302876.371830:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 11 pid: 6697 timeout: 813 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.371844:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 11 pid: 6697 timeout: 813 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.371866:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe43095184f498 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d91 expref: 11 pid: 6697 timeout: 813 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.371884:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302876.371888:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302876.371896:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.371906:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.371943:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.371957:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f754000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371171584:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302876.372097:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599480 going to sleep for 11 seconds 00000100:00100000:2.0:1789302876.386496:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.386768:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f754000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371171584:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302876.386803:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302876.386965:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350500096 00000100:00100000:1.0:1789302876.386973:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350500096 00002000:00100000:1.0:1789302876.387019:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1d:0x0] (0->157) req@ffff960f11de6300 x1876219350500096/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:452/0 lens 648/0 e 0 to 0 dl 1789302887 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302876.387029:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f11de6300 with x1876219350500096 ext(0->157) 00002000:00010000:1.0:1789302876.387044:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f11de6300 00000100:00100000:1.0:1789302876.387073:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 123 00000100:00100000:1.0:1789302876.387082:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350500096:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1789302876.387794:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350499968:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 16001us (16419us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.387804:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 425 00000100:00100000:1.0:1789302876.389425:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350500096:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 2336us (2929us total) trans 4294967377 rc 0/0 00000100:00100000:1.0:1789302876.389437:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 123 00002000:00100000:1.0:1789302876.389450:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1d:0x0] (0->157) req@ffff960f11de6300 x1876219350500096/t4294967377(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:452/0 lens 648/448 e 0 to 0 dl 1789302887 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302876.389459:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f11de6300 with x1876219350500096 ext(0->157) 00002000:00010000:1.0:1789302876.389465:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f11de6300 00000100:00100000:0.0:1789302876.394989:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.395168:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350500224 00000100:00100000:3.0:1789302876.395250:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350500224 00000100:00100000:3.0:1789302876.395358:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1415 00000100:00100000:3.0:1789302876.395369:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32145:x1876219350500224:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1789302876.439189:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968237 is committed 00000020:00080000:3.0:1789302876.439216:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 33 batchid updated 00000020:00080000:3.0:1789302876.439234:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 33 00000020:00080000:3.0:1789302876.439243:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 33 00000001:00080000:3.0:1789302876.439257:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968240 is committed 00000100:00100000:1.0:1789302876.439754:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32145:x1876219350500224:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 44382us (44769us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302876.439766:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1415 00000100:00100000:3.0:1789302876.444311:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.444364:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350500352 00000100:00100000:3.0:1789302876.444374:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350500352 00000100:00100000:3.0:1789302876.444426:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 402 00000100:00100000:3.0:1789302876.444437:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350500352:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000001:00080000:1.0:1789302876.496902:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967377 is committed 00000100:00100000:3.0:1789302876.497455:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350500352:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 53014us (53147us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.497466:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 402 00000100:00100000:3.0:1789302876.500688:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302876.500819:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350500608 00000100:00100000:2.0:1789302876.500829:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350500608 00000100:00100000:2.0:1789302876.500900:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1416 00000100:00100000:2.0:1789302876.500910:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32145:x1876219350500608:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:2.0:1789302876.500971:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302876.501010:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f491 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d8a expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.501037:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:2.0:1789302876.501043:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302876.501074:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.501087:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.501097:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302876.501495:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000adb81fd5 reply state 00000000962313e0 transno 4294968241 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302876.501569:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:2.0:1789302876.501671:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32145:x1876219350500608:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 761us (997us total) trans 4294968241 rc 0/0 00000100:00100000:2.0:1789302876.501677:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1416 00010000:00010000:1.0:1789302876.507496:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.507566:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.507574:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.507589:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302876.507603:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.507612:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.507631:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302876.507634:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302876.507639:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.507645:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4a6 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.509936:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302876.510465:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350500736 00000100:00100000:0.0:1789302876.510476:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350500736 00000100:00100000:0.0:1789302876.510572:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1417 00000100:00100000:0.0:1789302876.510583:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32145:x1876219350500736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302876.510619:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302876.510642:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0d9f expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.510706:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.510721:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.510740:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302876.510743:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302876.510747:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a800, rc 0) 00000100:00100000:0.0:1789302876.510933:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32145:x1876219350500736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 350us (1004us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302876.510940:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1417 00000100:00100000:0.0:1789302876.518259:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302876.518495:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350500864 00000100:00100000:2.0:1789302876.518674:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350500864 00000100:00100000:2.0:1789302876.519019:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 403 00000100:00100000:2.0:1789302876.519031:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350500864:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:2.0:1789302876.519308:0:6697:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:2.0:1789302876.519313:0:6697:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb mtime from disk: 0 -> 1789302867 00002000:00010000:2.0:1789302876.519316:0:6697:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb atime from disk: 0 -> 1789302867 00002000:00010000:2.0:1789302876.519317:0:6697:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb ctime from disk: 0 -> 1789302867 00002000:00010000:2.0:1789302876.519319:0:6697:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1d:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:2.0:1789302876.519386:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350500864:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 356us (1136us total) trans 4294967378 rc 0/0 00000100:00100000:2.0:1789302876.519394:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 403 00000100:00100000:2.0:1789302876.522408:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.522483:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350501120 00000100:00100000:1.0:1789302876.522493:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350501120 00000100:00100000:1.0:1789302876.522614:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1418 00000100:00100000:1.0:1789302876.522624:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32145:x1876219350501120:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1789302876.522945:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.522962:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.522972:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.523130:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.523145:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 52 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.523159:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f709880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371171712:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302876.526009:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e039ffb40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302876.529449:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.529547:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350501248 00000100:00100000:3.0:1789302876.529556:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350501248 00010000:00010000:3.0:1789302876.529607:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 53 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.529705:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 426 00000100:00100000:3.0:1789302876.529715:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2347:x1876219350501248:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302876.529730:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302876.529739:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 53 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.529752:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 53 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.529772:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302876.529778:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.529788:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.529796:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 5 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.529815:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302876.529871:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2347:x1876219350501248:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 157us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.529879:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 426 00010000:00010000:0.0:1789302876.530054:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f460 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2b:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d75 expref: 52 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302876.530075:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f709880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371171712:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302876.530096:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.530110:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.530115:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.530145:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.530153:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.530159:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.530164:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302876.530168:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.530175:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.531499:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302876.531511:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:0.0:1789302876.531518:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.531535:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f4b4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302876.531615:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32145:x1876219350501120:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 8991us (9215us total) trans 0 rc -22/-22 00000100:00100000:0.0:1789302876.531624:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1418 00000100:00100000:2.0:1789302876.536583:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.536794:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350501376 00000100:00100000:1.0:1789302876.536804:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350501376 00000100:00100000:1.0:1789302876.536901:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:1.0:1789302876.536911:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350501376:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302876.536956:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.536976:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.536984:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302876.536988:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302876.536992:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.537027:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.537042:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 11 pid: 6695 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.537062:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f709880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371171840:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302876.537143:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960e039ffc00 going to sleep for 11 seconds 00000100:00100000:0.0:1789302876.541167:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.541293:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f709880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371171840:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302876.541318:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302876.541450:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350501504 00000100:00100000:2.0:1789302876.541457:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350501504 00010000:00010000:2.0:1789302876.541491:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 12 pid: 6695 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302876.541569:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 427 00000100:00100000:2.0:1789302876.541580:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350501504:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302876.541594:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302876.541749:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 12 pid: 6695 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.541763:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 12 pid: 6695 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.541784:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe43095184f49f lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d98 expref: 12 pid: 6695 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.541799:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302876.541802:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302876.541807:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.541821:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.541850:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302876.541943:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350501504:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 364us (785us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302876.541951:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 427 00010000:00010000:1.0:1789302876.542003:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542012:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542038:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542056:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542064:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542070:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302876.542075:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542087:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542099:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.542106:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe43095184f4bb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302876.542151:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350501376:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 5242us (5655us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302876.542159:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 404 00000100:00100000:2.0:1789302876.556589:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302876.556741:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350501760 00000100:00100000:0.0:1789302876.556752:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350501760 00000100:00100000:0.0:1789302876.556813:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1419 00000100:00100000:0.0:1789302876.556824:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32145:x1876219350501760:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302876.556917:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.556936:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.556949:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.557191:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.557203:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.557211:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.557249:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.557262:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 52 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302876.557281:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371171968:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302876.557369:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302876.564458:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371171968:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302876.564488:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302876.568664:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.568928:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350501888 00000100:00100000:3.0:1789302876.568938:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350501888 00010000:00010000:3.0:1789302876.568975:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 53 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.569019:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 428 00000100:00100000:3.0:1789302876.569030:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2348:x1876219350501888:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302876.569046:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302876.569053:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 53 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.569066:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 53 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.569087:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184f4ad lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d9f expref: 53 pid: 6919 timeout: 813 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.569103:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302876.569108:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.569118:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.569125:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.569158:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302876.569217:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350501888:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 188us (562us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.569225:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 428 00010000:00010000:0.0:1789302876.569287:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.569296:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302876.569496:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968242 00000001:00080000:0.0:1789302876.569697:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968243 00000001:00080000:0.0:1789302876.569747:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968244 00000100:00100000:0.0:1789302876.569823:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371172096:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789302876.569874:0:18427:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302876.569913:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:1.0:1789302876.569996:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371172096 00000100:00100000:1.0:1789302876.570003:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371172096 00000100:00100000:1.0:1789302876.570067:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 71 00000100:00100000:1.0:1789302876.570075:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:18427:x1876219371172096:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789302876.570144:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:18427:x1876219371172096:12345-0@lo:1000:mdt00_004.0 Request processed in 70us (270us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302876.570150:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 71 00000100:00080000:3.0:1789302876.570790:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302876.571997:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371172096:0@lo:1000:mdt00_004.0 00000001:00080000:0.0:1789302876.572456:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968245 00000100:00100000:0.0:1789302876.572589:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371172224:0@lo:1000:mdt00_004.0 00000100:00100000:0.0:1789302876.573103:0:18427:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302876.573142:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e9c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302876.573164:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371172224 00000100:00100000:1.0:1789302876.573172:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371172224 00000100:00100000:1.0:1789302876.573239:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 72 00000100:00100000:1.0:1789302876.573249:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:18427:x1876219371172224:12345-0@lo:1000:mdt00_004.0 00000100:00100000:1.0:1789302876.573472:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:18427:x1876219371172224:12345-0@lo:1000:mdt00_004.0 Request processed in 224us (367us total) trans 4294967379 rc 0/0 00000100:00100000:1.0:1789302876.573482:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 72 00000100:00100000:0.0:1789302876.573577:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-mdtlov_UUID:18427:1876219371172224:0@lo:1000:mdt00_004.0 00000001:00080000:0.0:1789302876.573665:0:18427:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968246 00010000:00010000:0.0:1789302876.574749:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574765:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574772:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574778:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302876.574783:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574790:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574805:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302876.574809:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302876.574813:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574819:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184f4c9 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2b:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574852:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574858:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574863:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574869:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302876.574872:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574878:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40218400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574885:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:0.0:1789302876.574887:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302876.574895:0:18427:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574905:0:18427:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574914:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.574952:0:18427:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 103 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302876.574967:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371172352:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302876.575053:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e9c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302876.580835:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302876.580897:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290a680 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371172352:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302876.580932:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.580942:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184f3fe lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.580958:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350502016 00000100:00100000:3.0:1789302876.580966:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350502016 00010000:00010000:0.0:1789302876.580979:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.580992:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581003:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581012:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581018:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581024:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.581025:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 286 00010000:00010000:0.0:1789302876.581030:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00000100:00100000:3.0:1789302876.581034:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32142:x1876219350502016:12345-192.168.203.11@tcp:49:cat.0 00010000:00010000:0.0:1789302876.581034:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581041:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581052:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:0.0:1789302876.581054:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302876.581058:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581062:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184f4d0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2a:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581072:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581077:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581082:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581087:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302876.581089:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581095:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581101:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:0.0:1789302876.581103:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302876.581106:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.581111:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184f4c2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2a:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302876.581155:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32142:x1876219350502016:12345-192.168.203.11@tcp:49:cat.0 Request processed in 121us (328us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.581166:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 286 00000100:00100000:0.0:1789302876.581319:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32145:x1876219350501760:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 24497us (24738us total) trans 4294968247 rc 0/0 00000100:00100000:0.0:1789302876.581328:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1419 00000020:00080000:0.0:1789302876.581332:0:18427:0:(tgt_main.c:784:tgt_ses_key_exit()) total 6 transactions per RPC 00000040:00080000:3.0:1789302876.581825:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302876.581832:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302876.583946:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:3.0:1789302876.583966:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302876.593533:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.599742:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350502272 00000100:00100000:3.0:1789302876.599755:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350502272 00000100:00100000:3.0:1789302876.599865:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 287 00000100:00100000:3.0:1789302876.599877:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ea450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32145:x1876219350502272:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302876.601706:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302876.601783:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000004:00080000:3.0:1789302876.603869:0:12976:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x2b:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:3.0:1789302876.604023:0:12976:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968248 00010000:00010000:3.0:1789302876.604098:0:12976:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.604117:0:12976:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.604129:0:12976:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.604138:0:12976:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.604145:0:12976:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.604150:0:12976:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302876.604404:0:12976:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302876.604557:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ea450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:32145:x1876219350502272:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 4678us (11054us total) trans 4294968249 rc 0/0 00000100:00100000:3.0:1789302876.604567:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 287 00000020:00080000:3.0:1789302876.604573:0:12976:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00000100:00100000:2.0:1789302876.605901:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350502400 00000100:00100000:2.0:1789302876.605913:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350502400 00000100:00100000:2.0:1789302876.605975:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 405 00000100:00100000:2.0:1789302876.605986:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+14:32142:x1876219350502400:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302876.606003:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302876.606025:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184f4de lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0da6 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302876.606052:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1d:0x0] initial LVB size: 158, mtime: 0x6aa69853, atime: 0x6aa69853, ctime: 0x6aa69853, blocks: 0xb 00010000:00010000:2.0:1789302876.606071:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=713s v=(0 0) 00010000:00010000:2.0:1789302876.606076:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302876.606085:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184f4de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0da6 expref: 15 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.606103:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000070d624f2/0x64fe43095184f4de lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0da6 expref: 15 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302876.606115:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbfc00, rc 0) 00000100:00100000:2.0:1789302876.606166:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:32142:x1876219350502400:12345-192.168.203.11@tcp:101:cat.0 Request processed in 182us (4381us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302876.606173:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 405 00000100:00080000:3.0:1789302876.607307:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302876.608407:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350502656 00000100:00100000:0.0:1789302876.608419:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350502656 00000100:00100000:0.0:1789302876.608493:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 66 00000100:00100000:0.0:1789302876.608548:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32143:x1876219350502656:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:0.0:1789302876.608565:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302876.608591:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f405 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0d60 expref: 104 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302876.608618:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:0.0:1789302876.608629:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302876.608702:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:32143:x1876219350502656:12345-192.168.203.11@tcp:103:cat.0 Request processed in 190us (9520us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302876.608711:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 66 00010000:00010000:1.0:1789302876.612435:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302876.612452:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.612464:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302876.612469:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.612479:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.612498:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2b:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:1.0:1789302876.612516:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789302876.612532:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.612538:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f4d7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2b:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 12976 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.612564:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302876.612566:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302876.612568:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302876.622227:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.622397:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350502784 00000100:00100000:3.0:1789302876.622406:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350502784 00000100:00100000:3.0:1789302876.622462:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1420 00000100:00100000:3.0:1789302876.622472:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:32143:x1876219350502784:12345-192.168.203.11@tcp:58:cat.0 00000100:00100000:3.0:1789302876.622753:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:32143:x1876219350502784:12345-192.168.203.11@tcp:58:cat.0 Request processed in 280us (536us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302876.622761:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1420 00000100:00100000:3.0:1789302876.625732:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302876.625805:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350502912 00000100:00100000:1.0:1789302876.625814:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350502912 00000100:00100000:1.0:1789302876.625874:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1421 00000100:00100000:1.0:1789302876.625884:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+103:32143:x1876219350502912:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302876.625903:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302876.625927:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f4e5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dad expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.626001:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f4e5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dad expref: 104 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.626016:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9c00/0x64fe43095184f4e5 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2a:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dad expref: 104 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302876.626037:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2a:0x0].0x0 (ffff960f2c94dd00) 00010000:00010000:1.0:1789302876.626041:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302876.626044:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9c00, rc 0) 00000100:00100000:1.0:1789302876.626100:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32143:x1876219350502912:12345-192.168.203.11@tcp:101:cat.0 Request processed in 217us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302876.626107:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1421 00000100:00100000:1.0:1789302876.639784:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302876.639958:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350503040 00000100:00100000:0.0:1789302876.639969:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350503040 00002000:00100000:0.0:1789302876.640023:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1d:0x0] (0->4095) req@ffff960f3f708a80 x1876219350503040/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:452/0 lens 488/0 e 0 to 0 dl 1789302887 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302876.640034:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f708a80 with x1876219350503040 ext(0->4095) 00002000:00010000:0.0:1789302876.640042:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f708a80 00000100:00100000:0.0:1789302876.640123:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 124 00000100:00100000:0.0:1789302876.640133:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f708a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2345:x1876219350503040:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:0.0:1789302876.640364:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f708a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2345:x1876219350503040:12345-192.168.203.11@tcp:3:cat.0 Request processed in 232us (591us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302876.640371:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 124 00002000:00100000:0.0:1789302876.640378:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1d:0x0] (0->4095) req@ffff960f3f708a80 x1876219350503040/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:452/0 lens 488/4536 e 0 to 0 dl 1789302887 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302876.640385:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f708a80 with x1876219350503040 ext(0->4095) 00002000:00010000:0.0:1789302876.640389:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f708a80 00000100:00100000:1.0:1789302876.646151:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302876.646484:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350503296 00000100:00100000:3.0:1789302876.646493:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350503296 00000100:00100000:3.0:1789302876.646584:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 288 00000100:00100000:3.0:1789302876.646603:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32143:x1876219350503296:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302876.646828:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32143:x1876219350503296:12345-192.168.203.11@tcp:35:cat.0 Request processed in 234us (684us total) trans 4294968250 rc 0/0 00000100:00100000:3.0:1789302876.646836:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 288 00010000:00010000:1.0:1789302876.817841:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302877.570906:0:22232:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302877.570915:0:22232:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302877.570960:0:22232:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 43 count 3 00000040:00080000:0.0:1789302877.570972:0:22232:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:0.0:1789302877.652612:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302877.652620:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302877.653086:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1789302877.653116:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302877.837408:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302878.222591:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371172864:0@lo:400:kworker.0 00000100:00100000:3.0:1789302878.222657:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302878.223160:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1789302878.223169:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371172480:0@lo:400:kworker.0 00000100:00100000:2.0:1789302878.224485:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371172864 00000100:00100000:2.0:1789302878.224497:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371172864 00000100:00100000:2.0:1789302878.224587:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 518 00000100:00100000:2.0:1789302878.224599:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371172864:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302878.224897:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371172608:0@lo:400:kworker.0 00000100:00080000:0.0:1789302878.227105:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302878.227130:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371172736:0@lo:400:kworker.0 00000100:00100000:2.0:1789302878.232244:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371172864:12345-0@lo:400:kworker.0 Request processed in 7641us (9579us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302878.232261:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 518 00000100:00100000:1.0:1789302878.234455:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302878.234527:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371172608 00000100:00100000:1.0:1789302878.234537:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371172608 00000100:00100000:1.0:1789302878.234593:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:1.0:1789302878.234602:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371172608:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302878.234660:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371172608:12345-0@lo:400:kworker.0 Request processed in 59us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302878.234680:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:1.0:1789302878.234743:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e038daa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371172608:0@lo:400:kworker.0 00000100:00100000:1.0:1789302878.234755:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371173120:0@lo:400:kworker.0 00000100:00100000:1.0:1789302878.234804:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302878.243019:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371173120 00000100:00100000:2.0:1789302878.243033:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371173120 00000100:00100000:2.0:1789302878.243108:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 519 00000100:00100000:2.0:1789302878.243119:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371173120:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302878.243214:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371173120:12345-0@lo:400:kworker.0 Request processed in 97us (8408us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302878.243221:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 519 00000100:00080000:2.0:1789302878.243312:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302878.251139:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdc000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371173120:0@lo:400:kworker.0 00000100:00100000:0.0:1789302878.253727:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302878.253934:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371172736 00000100:00100000:2.0:1789302878.253945:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371172736 00000100:00100000:2.0:1789302878.254004:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:2.0:1789302878.254014:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371172736:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302878.254074:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3303:x1876219371172736:12345-0@lo:400:kworker.0 Request processed in 62us (354us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302878.254080:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00080000:2.0:1789302878.254100:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302878.254443:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302878.254479:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371172480 00000100:00100000:1.0:1789302878.254484:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371172480 00000100:00100000:3.0:1789302878.254494:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371172992:0@lo:400:kworker.0 00000100:00100000:1.0:1789302878.254559:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:1.0:1789302878.254568:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371172480:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302878.254595:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302878.254620:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3300:x1876219371172480:12345-0@lo:400:kworker.0 Request processed in 53us (178us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302878.254622:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371172992 00000100:00100000:3.0:1789302878.254628:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371172992 00000100:00100000:1.0:1789302878.254632:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00100000:1.0:1789302878.254834:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:1.0:1789302878.254841:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371172992:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302878.254871:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3300:x1876219371172992:12345-0@lo:400:kworker.0 Request processed in 32us (277us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302878.254877:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00080000:1.0:1789302878.254890:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302878.255364:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3300:1876219371172992:0@lo:400:kworker.0 00000100:00100000:3.0:1789302878.255379:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371172864:0@lo:400:kworker.0 00000100:00100000:3.0:1789302878.255402:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:4acabb58-ab41-4250-a3d6-ebb667f59134:3300:1876219371172480:0@lo:400:kworker.0 00000100:00100000:0.0:1789302878.255701:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3303:1876219371172736:0@lo:400:kworker.0 00000001:00080000:3.0:1789302878.298447:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967379 is committed 00000100:00100000:2.0:1789302878.734037:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1789302878.734063:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef78e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371173248:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302878.734140:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302878.734580:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371173248 00000100:00100000:1.0:1789302878.734589:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371173248 00000100:00100000:1.0:1789302878.734644:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:1.0:1789302878.734656:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371173248:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302878.734691:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302878.734757:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371173248:12345-0@lo:13:osp-pre-1-0.0 Request processed in 103us (627us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302878.734764:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:0.0:1789302878.734839:0:3302:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967379 gen 2 00000004:00080000:0.0:1789302878.734845:0:3302:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f70e300, transno 4294967379 00000100:00100000:0.0:1789302878.734874:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef78e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371173248:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302878.861556:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302879.244950:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302879.244984:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef79500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371173376:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302879.245073:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302879.245244:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371173376 00000100:00100000:1.0:1789302879.245256:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371173376 00000100:00100000:1.0:1789302879.245336:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 215 00000100:00100000:1.0:1789302879.245348:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371173376:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302879.245396:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302879.245765:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371173376:12345-0@lo:13:osp-pre-0-0.0 Request processed in 419us (690us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302879.245774:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 215 00000100:00100000:0.0:1789302879.253141:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef79500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371173376:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302879.503705:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302879.503822:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302879.503854:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302879.503956:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302879.503983:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302879.504114:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350503552 00000100:00100000:2.0:1789302879.504128:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350503552 00000100:00100000:2.0:1789302879.504228:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:2.0:1789302879.504242:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:2346:x1876219350503552:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302879.504350:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:2346:x1876219350503552:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 112us (660us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302879.504358:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:2.0:1789302879.504583:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350503424 00000100:00100000:2.0:1789302879.504589:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350503424 00000100:00100000:2.0:1789302879.504619:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00100000:2.0:1789302879.504626:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350503424:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302879.504661:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350503424:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (705us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302879.504668:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00080000:2.0:1789302879.504742:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302879.505521:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302879.505591:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302879.505631:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350503808 00000100:00100000:0.0:1789302879.505640:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350503808 00000100:00100000:0.0:1789302879.505697:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504064 00000100:00100000:0.0:1789302879.505701:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504064 00000100:00100000:0.0:1789302879.505722:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350503680 00000100:00100000:0.0:1789302879.505725:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350503680 00000100:00100000:0.0:1789302879.505752:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 520 00000100:00100000:0.0:1789302879.505762:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350503808:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302879.505827:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350503808:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 67us (2004us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302879.505835:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 520 00000100:00100000:0.0:1789302879.505851:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 521 00000100:00100000:0.0:1789302879.505858:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350504064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302879.505885:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350504064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (2031us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302879.505892:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 521 00000100:00100000:0.0:1789302879.505903:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 522 00000100:00100000:0.0:1789302879.505909:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2346:x1876219350503680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302879.505934:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2346:x1876219350503680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (1951us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302879.505940:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 522 00000100:00100000:0.0:1789302879.506408:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350503936 00000100:00100000:0.0:1789302879.506415:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350503936 00000100:00100000:0.0:1789302879.506460:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00100000:0.0:1789302879.506467:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:2346:x1876219350503936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302879.506555:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:2346:x1876219350503936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 87us (1060us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302879.506562:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00080000:2.0:1789302879.506872:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302879.506878:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302879.506882:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302879.507861:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504192 00000100:00100000:0.0:1789302879.507874:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504192 00000100:00100000:0.0:1789302879.508146:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 523 00000100:00100000:0.0:1789302879.508158:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e038daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350504192:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302879.508266:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e038daa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350504192:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 110us (2674us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302879.508275:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 523 00000100:00080000:2.0:1789302879.508604:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1789302879.890879:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302880.723141:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302880.723154:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302880.723552:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 43 count 2 00000040:00080000:0.0:1789302880.723580:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302880.908957:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789302881.899625:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968247 is committed 00000020:00080000:0.0:1789302881.899649:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f07e4e480 refcount 2 committed 0 result 0 batchid 34 00000020:00080000:0.0:1789302881.899655:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f12140a00 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f07e4e6c0 00000020:00080000:0.0:1789302881.899659:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f12140000 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3f00e000 00000001:00080000:0.0:1789302881.899707:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968250 is committed 00000001:00080000:0.0:1789302881.899714:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968249 is committed 00000020:00080000:0.0:1789302881.899728:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 34 committed batchid 33 00000020:00080000:0.0:1789302881.899956:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 33 00000001:00080000:1.0:1789302881.910342:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967379 is committed 00000001:00080000:1.0:1789302881.913748:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967378 is committed 00010000:00010000:1.0:1789302881.932630:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302882.393667:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302882.395251:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504320 00000100:00100000:3.0:1789302882.395268:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504320 00000100:00100000:3.0:1789302882.395359:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1422 00000100:00100000:3.0:1789302882.395373:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32115:x1876219350504320:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:3.0:1789302882.395550:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32115:x1876219350504320:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 180us (1899us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302882.395570:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1422 00000100:00080000:2.0:1789302882.395777:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302882.957193:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302883.342602:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371173888:0@lo:400:kworker.0 00000100:00100000:2.0:1789302883.342707:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302883.342885:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371173632:0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.342923:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302883.342968:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371174144:0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.342990:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302883.343019:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789302883.343025:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371173760:0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.343045:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302883.343079:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371173632 00000100:00100000:0.0:1789302883.343087:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371173632 00000100:00100000:0.0:1789302883.343135:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371173760 00000100:00100000:0.0:1789302883.343138:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371173760 00000100:00100000:0.0:1789302883.343171:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:0.0:1789302883.343181:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371173632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.343240:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371173632:12345-0@lo:400:kworker.0 Request processed in 60us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302883.343246:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:0.0:1789302883.343266:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:0.0:1789302883.343272:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371173760:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.343299:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3302:x1876219371173760:12345-0@lo:400:kworker.0 Request processed in 28us (253us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302883.343304:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:0.0:1789302883.343323:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371173888 00000100:00100000:0.0:1789302883.343326:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371173888 00000100:00100000:0.0:1789302883.343352:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371174144 00000100:00100000:0.0:1789302883.343355:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371174144 00000100:00100000:0.0:1789302883.343375:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 524 00000100:00100000:0.0:1789302883.343381:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371173888:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.343412:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371173888:12345-0@lo:400:kworker.0 Request processed in 32us (706us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302883.343418:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 524 00000100:00100000:0.0:1789302883.343430:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1789302883.343436:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371174144:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302883.343444:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371173504:0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.343462:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371174144:12345-0@lo:400:kworker.0 Request processed in 27us (472us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302883.343467:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00000100:00100000:3.0:1789302883.343494:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789302883.343519:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302883.343524:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302883.343528:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302883.343563:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3302:1876219371173760:0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.343574:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371173632:0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.343582:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371174144:0@lo:400:kworker.0 00000100:00100000:3.0:1789302883.343616:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcdfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371174016:0@lo:400:kworker.0 00000100:00100000:3.0:1789302883.343643:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302883.343679:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371173504 00000100:00100000:3.0:1789302883.343683:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371173504 00000100:00100000:3.0:1789302883.343710:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 619 00000100:00100000:3.0:1789302883.343716:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371173504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302883.343749:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3301:x1876219371173504:12345-0@lo:400:kworker.0 Request processed in 33us (255us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302883.343759:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 619 00000100:00100000:3.0:1789302883.343786:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371174016 00000100:00100000:3.0:1789302883.343790:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371174016 00000100:00100000:3.0:1789302883.343818:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:3.0:1789302883.343824:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371174016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302883.343837:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967379 gen 2 00000004:00080000:2.0:1789302883.343843:0:3300:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967379 committed transno 4294967379 00000100:00100000:3.0:1789302883.343851:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371174016:12345-0@lo:400:kworker.0 Request processed in 28us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302883.343856:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:2.0:1789302883.343861:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371173888:0@lo:400:kworker.0 00000100:00080000:3.0:1789302883.343871:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302883.343887:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcdfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371174016:0@lo:400:kworker.0 00000100:00100000:3.0:1789302883.343895:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:4acabb58-ab41-4250-a3d6-ebb667f59134:3301:1876219371173504:0@lo:400:kworker.0 00000100:00100000:0.0:1789302883.853159:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371174272:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302883.853323:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302883.853992:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371174272 00000100:00100000:3.0:1789302883.854006:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371174272 00000100:00100000:3.0:1789302883.854160:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:3.0:1789302883.854172:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371174272:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302883.854220:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302883.854296:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371174272:12345-0@lo:13:osp-pre-1-0.0 Request processed in 126us (973us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302883.854304:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:0.0:1789302883.854412:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ad880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371174272:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302883.984397:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789302884.107363:0:22433:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12t: Multiple parallel reads for a HSM imported file ========================================================== 08:34:42 (1789302882) 00000100:00100000:2.0:1789302884.367906:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371174400:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302884.368010:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302884.368141:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371174400 00000100:00100000:3.0:1789302884.368154:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371174400 00000100:00100000:3.0:1789302884.368223:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 217 00000100:00100000:3.0:1789302884.368234:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371174400:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302884.368281:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 88408064 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302884.370942:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371174400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 2695us (2917us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302884.370961:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 217 00000100:00100000:2.0:1789302884.372295:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371174400:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302884.626880:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302884.627061:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504448 00000100:00100000:2.0:1789302884.627075:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504448 00000100:00100000:2.0:1789302884.627156:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 620 00000100:00100000:2.0:1789302884.627168:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350504448:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302884.627278:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350504448:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 111us (557us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302884.627286:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 620 00000100:00100000:1.0:1789302884.646775:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.646826:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.646866:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.646900:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.646919:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.646945:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302884.647145:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504704 00000100:00100000:0.0:1789302884.647159:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504704 00000100:00100000:0.0:1789302884.647243:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350505216 00000100:00100000:0.0:1789302884.647247:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350505216 00000100:00100000:0.0:1789302884.647278:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504832 00000100:00100000:0.0:1789302884.647281:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504832 00000100:00100000:0.0:1789302884.647298:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350505088 00000100:00100000:0.0:1789302884.647301:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350505088 00000100:00100000:0.0:1789302884.647319:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 526 00000100:00100000:0.0:1789302884.647330:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2347:x1876219350504704:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302884.647421:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:2347:x1876219350504704:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 93us (672us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302884.647429:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 526 00000100:00100000:0.0:1789302884.647448:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 527 00000100:00100000:0.0:1789302884.647454:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350505216:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302884.647731:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350505216:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 275us (863us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302884.647739:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 527 00000100:00100000:0.0:1789302884.647752:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 528 00000100:00100000:0.0:1789302884.647758:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350504832:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302884.647792:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350504832:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (872us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302884.647798:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 528 00000100:00100000:0.0:1789302884.647808:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 529 00000100:00100000:0.0:1789302884.647814:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350505088:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302884.647833:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350505088:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (888us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302884.647839:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 529 00000100:00100000:0.0:1789302884.647859:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504960 00000100:00100000:0.0:1789302884.647863:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504960 00000100:00100000:0.0:1789302884.647890:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350504576 00000100:00100000:0.0:1789302884.647892:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350504576 00000100:00100000:0.0:1789302884.647909:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 505 00000100:00100000:0.0:1789302884.647914:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350504960:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302884.647948:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350504960:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (1122us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302884.647954:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 505 00000100:00100000:0.0:1789302884.647966:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00100000:0.0:1789302884.647972:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350504576:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302884.648004:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:2347:x1876219350504576:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (1103us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302884.648010:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00080000:1.0:1789302884.649128:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302884.649137:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302884.649139:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302884.825407:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.827363:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350505344 00000100:00100000:1.0:1789302884.827385:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350505344 00000100:00100000:1.0:1789302884.827487:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1423 00000100:00100000:1.0:1789302884.827566:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32757:x1876219350505344:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:1.0:1789302884.827715:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32757:x1876219350505344:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 215us (2322us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302884.827724:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1423 00000100:00100000:0.0:1789302884.862057:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302884.862257:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350505472 00000100:00100000:2.0:1789302884.862270:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350505472 00000100:00100000:2.0:1789302884.862345:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1424 00000100:00100000:2.0:1789302884.862358:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32758:x1876219350505472:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302884.862387:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302884.862418:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f4ec lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0db4 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.862555:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.862574:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.862592:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863031:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863054:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863067:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863279:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863289:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863296:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863303:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302884.863308:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863316:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863328:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:2.0:1789302884.863331:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302884.863336:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863343:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f4f3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863356:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863362:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863368:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863373:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302884.863376:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863381:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863389:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff960f10d3c000) 00010000:00010000:2.0:1789302884.863392:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302884.863395:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863401:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f4fa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863446:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f4ec lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0db4 expref: 105 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863457:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:2.0:1789302884.863459:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302884.863475:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f4ec lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0db4 expref: 105 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.863485:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302884.863704:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32758:x1876219350505472:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1347us (1669us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302884.863713:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1424 00000100:00100000:2.0:1789302884.874715:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302884.875090:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350505728 00000100:00100000:3.0:1789302884.875100:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350505728 00000100:00100000:3.0:1789302884.875161:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1425 00000100:00100000:3.0:1789302884.875172:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32758:x1876219350505728:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:3.0:1789302884.875226:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302884.875251:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f23e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cbf expref: 104 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.875282:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:3.0:1789302884.875286:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302884.875570:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.875585:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.875593:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.875626:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 48 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.875642:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 48 pid: 6919 timeout: 821 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302884.875661:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371174528:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302884.875756:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75000 going to sleep for 11 seconds 00000100:00100000:0.0:1789302884.880629:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.880875:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350505856 00000100:00100000:1.0:1789302884.880885:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350505856 00010000:00010000:1.0:1789302884.880926:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 49 pid: 6919 timeout: 821 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302884.880976:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 429 00000100:00100000:1.0:1789302884.880986:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd07c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350505856:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302884.880999:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302884.881006:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 49 pid: 6919 timeout: 821 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.881019:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 49 pid: 6919 timeout: 821 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.881040:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:1.0:1789302884.881046:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.881056:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.881064:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.881088:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302884.881144:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd07c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2348:x1876219350505856:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 159us (524us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302884.881151:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 429 00010000:00010000:3.0:1789302884.881618:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f27d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0cd4 expref: 48 pid: 6919 timeout: 821 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302884.881636:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371174528:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302884.881666:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.881676:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.881682:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.881705:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.881714:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.881722:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302884.887014:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 16384 00010000:00010000:3.0:1789302884.887031:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.887042:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.887051:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.887057:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302884.887062:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.887070:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.887086:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f2c94d600) 00010000:00010000:3.0:1789302884.887089:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302884.887094:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.887099:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f501 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302884.887119:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dc17b65f reply state 0000000058c73149 transno 4294968251 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302884.887128:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302884.887236:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32758:x1876219350505728:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 12063us (12537us total) trans 4294968251 rc 0/0 00000100:00100000:3.0:1789302884.887245:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1425 00010000:00010000:1.0:1789302884.892937:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.892953:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.892962:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.892968:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302884.892973:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.892980:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.892994:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff960f0cdf2100) 00010000:00010000:1.0:1789302884.892997:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302884.893002:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.893008:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f508 lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302884.902199:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302884.902314:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350505984 00000100:00100000:2.0:1789302884.902325:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350505984 00000100:00100000:2.0:1789302884.902378:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1426 00000100:00100000:2.0:1789302884.902388:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+104:32759:x1876219350505984:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302884.902412:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302884.902434:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f50f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dbb expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.902551:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f516 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.902570:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f516 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.902581:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f516 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302884.902618:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:2.0:1789302884.902622:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302884.902640:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f516 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.902669:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f50f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dbb expref: 106 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.902689:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f516 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dbb expref: 105 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.902697:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:2.0:1789302884.902700:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302884.902704:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbec00, rc 0) 00000100:00100000:2.0:1789302884.902762:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32759:x1876219350505984:12345-192.168.203.11@tcp:101:bash.0 Request processed in 376us (572us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302884.902769:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1426 00000100:00100000:2.0:1789302884.910297:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302884.910394:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350506112 00000100:00100000:0.0:1789302884.910404:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350506112 00000100:00100000:0.0:1789302884.910462:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1427 00000100:00100000:0.0:1789302884.910482:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32759:x1876219350506112:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302884.910528:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302884.910549:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f51d lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dc2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910612:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910627:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910639:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910664:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910672:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910678:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910769:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ba00/0x64fe43095184f532 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910778:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ba00/0x64fe43095184f532 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910784:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ba00/0x64fe43095184f532 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302884.910810:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 16384 00000004:00200000:0.0:1789302884.910813:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302884.910829:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ba00/0x64fe43095184f532 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910840:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910846:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910852:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910857:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302884.910861:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910868:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910879:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302884.910881:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302884.910886:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910896:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f524 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910911:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910916:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910922:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910926:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302884.910928:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910933:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910939:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff960f09ae1c00) 00010000:00010000:0.0:1789302884.910942:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302884.910944:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910950:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f52b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910979:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f51d lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dc2 expref: 107 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.910993:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ba00/0x64fe43095184f532 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dc2 expref: 106 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.911000:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302884.911001:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1789302884.911004:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ba00, rc 0) 00000100:00100000:0.0:1789302884.911053:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32759:x1876219350506112:12345-192.168.203.11@tcp:101:bash.0 Request processed in 573us (765us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302884.911060:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1427 00000100:00100000:1.0:1789302884.913140:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302884.913199:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350506240 00000100:00100000:3.0:1789302884.913209:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350506240 00000100:00100000:3.0:1789302884.913280:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1428 00000100:00100000:3.0:1789302884.913290:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32759:x1876219350506240:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302884.913311:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302884.913330:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f539 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dc9 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913395:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913408:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913418:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913432:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913438:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913445:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913563:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913571:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913578:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913583:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302884.913589:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913597:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913609:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302884.913613:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302884.913618:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913624:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f540 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913636:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913643:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913648:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913653:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302884.913656:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913662:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913669:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302884.913671:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302884.913676:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913681:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f547 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913710:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f539 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dc9 expref: 107 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913719:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302884.913722:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302884.913725:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f539 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dc9 expref: 107 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.913734:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302884.913795:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32759:x1876219350506240:12345-192.168.203.11@tcp:101:bash.0 Request processed in 506us (662us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302884.913802:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1428 00000100:00100000:2.0:1789302884.950860:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302884.951154:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350506368 00000100:00100000:0.0:1789302884.951164:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350506368 00000100:00100000:0.0:1789302884.951220:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1429 00000100:00100000:0.0:1789302884.951231:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+48:32760:x1876219350506368:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302884.951249:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302884.951279:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f54e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dd0 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.951350:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.951364:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.951375:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302884.951410:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302884.951414:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302884.951434:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.951461:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f54e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 50 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.951492:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 49 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.951607:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302884.951613:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302884.951616:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ac00, rc 0) 00000100:00100000:1.0:1789302884.951820:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32760:x1876219350506368:12345-192.168.203.11@tcp:101:rm.0 Request processed in 589us (967us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302884.951829:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1429 00000100:00100000:2.0:1789302884.956882:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302884.956960:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350506496 00000100:00100000:1.0:1789302884.956969:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350506496 00000100:00100000:1.0:1789302884.957027:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1430 00000100:00100000:1.0:1789302884.957036:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:32760:x1876219350506496:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302884.957057:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302884.957077:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f55c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dd7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957154:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957168:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957179:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957192:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957199:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957204:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957303:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f571 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957315:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f571 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957322:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f571 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302884.957348:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 16384 00000004:00200000:1.0:1789302884.957351:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302884.957367:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f571 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957378:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957385:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957391:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957396:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302884.957400:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957407:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957418:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:1.0:1789302884.957421:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302884.957426:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957435:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f563 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957448:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957456:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957461:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957474:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302884.957477:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957483:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957490:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xef4d5f1f (ffff960f2c94df00) 00010000:00010000:1.0:1789302884.957492:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302884.957495:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957542:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f56a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xef4d5f1f bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957573:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f55c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd7 expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957586:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9a00/0x64fe43095184f571 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd7 expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302884.957595:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302884.957598:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:1.0:1789302884.957600:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9a00, rc 0) 00000100:00100000:1.0:1789302884.957647:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32760:x1876219350506496:12345-192.168.203.11@tcp:101:rm.0 Request processed in 613us (773us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302884.957655:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1430 00000100:00100000:0.0:1789302884.962795:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302884.963427:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350506624 00000100:00100000:3.0:1789302884.963439:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350506624 00000100:00100000:3.0:1789302884.963594:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1431 00000100:00100000:3.0:1789302884.963605:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32760:x1876219350506624:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302884.963624:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302884.963645:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f578 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dde expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965294:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965322:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965333:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965349:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965356:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965362:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965417:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965424:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965430:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965436:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302884.965439:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965446:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965456:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302884.965459:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302884.965477:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965487:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f57f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965522:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965528:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965534:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965552:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302884.965554:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965560:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965567:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f10d3c500) 00010000:00010000:2.0:1789302884.965569:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302884.965572:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965578:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f586 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965614:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f578 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dde expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965625:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302884.965627:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302884.965631:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184f578 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dde expref: 51 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302884.965640:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:2.0:1789302884.965712:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32760:x1876219350506624:12345-192.168.203.11@tcp:101:rm.0 Request processed in 2108us (2925us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302884.965721:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1431 00000100:00100000:3.0:1789302884.969687:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302884.969845:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350506752 00000100:00100000:0.0:1789302884.969856:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350506752 00000100:00100000:0.0:1789302884.969923:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1432 00000100:00100000:0.0:1789302884.969933:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32760:x1876219350506752:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302884.969954:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302884.969972:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f58d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0de5 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970038:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970051:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970061:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970074:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970081:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970086:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970139:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970147:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970153:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970160:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302884.970164:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970171:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970183:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302884.970186:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302884.970191:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970197:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f594 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970212:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970219:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970224:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970228:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302884.970231:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970236:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970243:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f09ae1c00) 00010000:00010000:0.0:1789302884.970245:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302884.970247:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970253:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f59b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970287:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f58d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0de5 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970296:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302884.970298:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302884.970302:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f58d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0de5 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302884.970314:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ae00, rc 0) 00000100:00100000:0.0:1789302884.970380:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32760:x1876219350506752:12345-192.168.203.11@tcp:101:rm.0 Request processed in 448us (702us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302884.970386:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1432 00000100:00100000:0.0:1789302884.993258:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302884.993345:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350506880 00000100:00100000:3.0:1789302884.993355:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350506880 00000100:00100000:3.0:1789302884.993418:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1433 00000100:00100000:3.0:1789302884.993429:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32761:x1876219350506880:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302884.993449:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302884.993483:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f5a2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dec expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993600:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993615:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993627:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993642:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993649:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993656:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993712:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993720:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993728:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993739:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302884.993744:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993753:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993765:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302884.993768:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302884.993773:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993779:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184f5a9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993793:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993800:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993806:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993811:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302884.993813:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993819:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993830:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302884.993833:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302884.993835:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993841:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f5b0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993874:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f5a2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dec expref: 51 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993883:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302884.993886:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302884.993889:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f5a2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dec expref: 51 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302884.993898:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302884.993952:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32761:x1876219350506880:12345-192.168.203.11@tcp:101:cp.0 Request processed in 525us (703us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302884.993959:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1433 00000100:00100000:3.0:1789302884.997329:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302884.998480:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350507008 00000100:00100000:0.0:1789302884.998491:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350507008 00000100:00100000:0.0:1789302884.998578:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1434 00000100:00100000:0.0:1789302884.998587:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32761:x1876219350507008:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302884.998608:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302884.998627:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f5b7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0df3 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001008:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001030:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001042:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001062:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001069:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001076:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001358:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001366:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001376:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001382:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302885.001386:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001393:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001404:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302885.001407:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302885.001411:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001418:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5be lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001431:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001437:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001442:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001446:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302885.001449:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001457:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001475:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94df00) 00010000:00010000:1.0:1789302885.001478:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302885.001481:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001487:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f5c5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001559:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f5b7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0df3 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001571:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302885.001574:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302885.001577:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f5b7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0df3 expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.001589:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345aa00, rc 0) 00000100:00100000:1.0:1789302885.001676:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32761:x1876219350507008:12345-192.168.203.11@tcp:101:cp.0 Request processed in 3089us (4354us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302885.001685:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1434 00010000:00010000:1.0:1789302885.004788:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302885.012591:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302885.012639:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350507264 00000100:00100000:2.0:1789302885.012651:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350507264 00000100:00100000:2.0:1789302885.012710:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1435 00000100:00100000:2.0:1789302885.012721:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32761:x1876219350507264:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302885.012741:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302885.012761:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f5cc lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0dfa expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.012880:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.012896:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.012909:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.012924:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.012932:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.012938:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302885.013636:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1d:0x0], index 1: 0 00010000:00010000:2.0:1789302885.013821:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013832:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013842:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013877:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013884:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013890:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013896:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302885.013900:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013907:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013924:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302885.013927:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302885.013932:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.013937:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095184f5d3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302885.013949:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009c0fe5d1 reply state 000000003a81e985 transno 4294968252 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302885.013956:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302885.013988:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f5cc lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 53 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.014003:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 52 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.014011:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302885.014013:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302885.014015:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:2.0:1789302885.014095:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32761:x1876219350507264:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1376us (1515us total) trans 4294968252 rc 0/0 00000100:00100000:2.0:1789302885.014103:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1435 00010000:00010000:1.0:1789302885.018114:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.018132:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.018141:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.018148:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302885.018155:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.018162:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.018190:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f10d3c500) 00010000:00010000:1.0:1789302885.018193:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302885.018197:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.018203:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184f5da lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302885.019637:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.024856:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350507392 00000100:00100000:1.0:1789302885.024871:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350507392 00000100:00100000:1.0:1789302885.024937:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 406 00000100:00100000:1.0:1789302885.024947:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350507392:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302885.024965:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302885.024988:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e01 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302885.025016:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1d:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789302885.025036:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=722s v=(0 0) 00010000:00010000:1.0:1789302885.025041:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302885.025051:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 10 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.025068:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 10 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.025082:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc 0) 00000100:00100000:1.0:1789302885.025155:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350507392:12345-192.168.203.11@tcp:101:cp.0 Request processed in 209us (5525us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302885.025162:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 406 00000100:00100000:0.0:1789302885.028960:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302885.029065:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350507520 00000100:00100000:2.0:1789302885.029075:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350507520 00000100:00100000:2.0:1789302885.029134:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 407 00000100:00100000:2.0:1789302885.029146:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:32761:x1876219350507520:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302885.029164:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302885.029195:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e08 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.029227:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.029235:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302885.029239:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302885.029245:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.029275:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.029289:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 11 pid: 8571 timeout: 822 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302885.029303:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371174656:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302885.029399:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599f00 going to sleep for 11 seconds 00000100:00100000:2.0:1789302885.031941:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371174656:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302885.031975:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 11 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.031996:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:2.0:1789302885.032056:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:32761:x1876219350507520:12345-192.168.203.11@tcp:101:cp.0 Request processed in 2913us (3104us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302885.032065:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 407 00000100:00100000:3.0:1789302885.036349:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302885.036599:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350507648 00000100:00100000:0.0:1789302885.036611:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350507648 00010000:00010000:0.0:1789302885.036663:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 11 pid: 8571 timeout: 822 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302885.036710:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 430 00000100:00100000:0.0:1789302885.036720:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350507648:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302885.036735:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302885.036759:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 11 pid: 8571 timeout: 822 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.036773:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 11 pid: 8571 timeout: 822 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.036795:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002c3fd20a/0x64fe43095184f5ef lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e01 expref: 11 pid: 8571 timeout: 822 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.036812:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302885.036816:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302885.036823:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.036833:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.036870:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302885.036884:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371174784:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302885.036962:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fda240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302885.048230:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.048329:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350507904 00000100:00100000:1.0:1789302885.048339:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350507904 00000100:00100000:1.0:1789302885.048401:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 289 00000100:00100000:1.0:1789302885.048411:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32761:x1876219350507904:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:1.0:1789302885.048802:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32761:x1876219350507904:12345-192.168.203.11@tcp:35:cp.0 Request processed in 392us (579us total) trans 4294968253 rc 0/0 00000100:00100000:1.0:1789302885.048810:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 289 00000100:00100000:3.0:1789302885.049352:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371174784:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302885.049374:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302885.049456:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350507648:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 12736us (13116us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302885.049481:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 430 00000100:00100000:0.0:1789302885.079004:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.079127:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350508032 00000100:00100000:1.0:1789302885.079136:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350508032 00000100:00100000:1.0:1789302885.079199:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1436 00000100:00100000:1.0:1789302885.079209:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32764:x1876219350508032:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302885.079231:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302885.079251:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e0f expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079346:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079360:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079371:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079385:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079393:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079399:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079498:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f612 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079555:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f612 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079563:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f612 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302885.079596:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2c:0x0]: returning size 158 00000004:00200000:1.0:1789302885.079601:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302885.079616:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f612 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079630:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f612 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079638:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079644:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079852:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079860:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302885.079864:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079872:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079883:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302885.079885:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302885.079890:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079896:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f604 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079910:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079918:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079929:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079933:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302885.079935:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079941:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079947:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d200) 00010000:00010000:1.0:1789302885.079949:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302885.079951:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079957:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f60b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.079988:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f5fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e0f expref: 108 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.080001:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f612 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e0f expref: 107 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.080009:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:1.0:1789302885.080011:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302885.080013:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8a00, rc 0) 00000100:00100000:1.0:1789302885.080077:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32764:x1876219350508032:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 869us (1080us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302885.080087:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1436 00000100:00100000:0.0:1789302885.085115:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302885.085398:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350508160 00000100:00100000:3.0:1789302885.085408:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350508160 00000100:00100000:3.0:1789302885.085526:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 408 00000100:00100000:3.0:1789302885.085542:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350508160:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302885.085560:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302885.085577:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f619 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e16 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302885.085629:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371174912:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302885.085707:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75000 going to sleep for 11 seconds 00002000:00010000:3.0:1789302885.090322:0:18578:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1789302885.090331:0:18578:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb mtime: 0 -> 1789302884 00002000:00010000:3.0:1789302885.090334:0:18578:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb atime: 0 -> 1789302884 00002000:00010000:3.0:1789302885.090336:0:18578:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x1d:0x0] updating lvb ctime: 0 -> 1789302884 00000100:00100000:3.0:1789302885.090362:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371174912:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302885.090399:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f619 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e16 expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.090422:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184f619 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e16 expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.090439:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302885.090634:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350508160:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 5098us (5526us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302885.090643:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 408 00000100:00100000:0.0:1789302885.095312:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.095418:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350508288 00000100:00100000:1.0:1789302885.095428:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350508288 00000100:00100000:1.0:1789302885.095574:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1437 00000100:00100000:1.0:1789302885.095585:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32764:x1876219350508288:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302885.095605:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302885.095624:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f620 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e1d expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.095698:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f627 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.095713:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f627 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.095723:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f627 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302885.095756:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 16384 00000004:00200000:1.0:1789302885.095759:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302885.095773:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f627 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.095797:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f620 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e1d expref: 109 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.095815:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f627 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e1d expref: 108 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.095840:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302885.095843:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302885.095847:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8000, rc 0) 00000100:00100000:1.0:1789302885.095908:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:32764:x1876219350508288:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 325us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302885.095915:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1437 00000100:00100000:0.0:1789302885.099716:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.099869:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350508416 00000100:00100000:1.0:1789302885.099877:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350508416 00000100:00100000:1.0:1789302885.099926:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 409 00000100:00100000:1.0:1789302885.099935:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350508416:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302885.099952:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302885.099970:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe43095184f62e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e24 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302885.100030:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371175040:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302885.100361:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335be180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302885.110049:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371175040:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302885.110091:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe43095184f62e lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e24 expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.110115:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe43095184f62e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e24 expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.110149:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9400, rc 0) 00000100:00100000:1.0:1789302885.110206:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350508416:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 10270us (10498us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302885.110216:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 409 00000100:00100000:2.0:1789302885.116225:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302885.116305:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350508672 00000100:00100000:3.0:1789302885.116315:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350508672 00000100:00100000:3.0:1789302885.116374:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1438 00000100:00100000:3.0:1789302885.116384:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:32764:x1876219350508672:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:2.0:1789302885.116991:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].43, 136 off14040 00000100:00100000:2.0:1789302885.117083:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:32764:x1876219350508672:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 700us (868us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302885.117092:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1438 00000040:00080000:0.0:1789302885.836876:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302885.836891:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302885.838892:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 44 count 3 00000040:00080000:0.0:1789302885.838914:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302885.840933:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63c000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371175168:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302885.841454:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ab25480 going to sleep for 11 seconds 00000100:00100000:0.0:1789302885.846881:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63c000 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371175168:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:2.0:1789302885.852915:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.853589:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350508800 00000100:00100000:1.0:1789302885.853604:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350508800 00000100:00100000:1.0:1789302885.853677:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1439 00000100:00100000:1.0:1789302885.853688:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32774:x1876219350508800:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302885.853736:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302885.853869:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32774:x1876219350508800:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 182us (971us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302885.853877:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1439 00000100:00100000:2.0:1789302885.864827:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.864928:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350509056 00000100:00100000:1.0:1789302885.864942:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350509056 00000100:00100000:1.0:1789302885.865017:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1440 00000100:00100000:1.0:1789302885.865029:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32774:x1876219350509056:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302885.865059:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302885.865088:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f635 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e2b expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.865238:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f63c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.865257:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f63c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.865271:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f63c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302885.865289:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2c:0x0]: returning size 158 00010000:00010000:1.0:1789302885.865472:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f635 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e2b expref: 56 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.865495:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f63c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e2b expref: 55 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.865546:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:1.0:1789302885.865549:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302885.865554:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9600, rc 0) 00000100:00100000:1.0:1789302885.865633:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32774:x1876219350509056:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 606us (821us total) trans 4294968254 rc 0/0 00000100:00100000:1.0:1789302885.865640:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1440 00000100:00100000:3.0:1789302885.875753:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302885.875798:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302885.875835:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350509312 00000100:00100000:3.0:1789302885.875848:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350509312 00000100:00100000:3.0:1789302885.875933:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 67 00000100:00100000:3.0:1789302885.875945:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2347:x1876219350509312:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302885.875965:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302885.875996:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f63c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e2b expref: 55 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.876020:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:3.0:1789302885.876027:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302885.876091:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2347:x1876219350509312:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 148us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302885.876100:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 67 00000100:00100000:1.0:1789302885.878639:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350509184 00000100:00100000:1.0:1789302885.878657:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350509184 00000100:00100000:1.0:1789302885.879243:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 410 00000100:00100000:1.0:1789302885.879259:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350509184:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302885.879324:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.879353:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.879363:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302885.879369:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302885.879374:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.879425:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 10 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.879441:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 10 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302885.880838:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371175296:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302885.881102:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f335be180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302885.896020:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ad500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371175296:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302885.896067:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302885.897211:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302885.897342:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350509440 00000100:00100000:0.0:1789302885.897355:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350509440 00002000:00100000:0.0:1789302885.897415:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1d:0x0] (0->157) req@ffff960f3ef31f80 x1876219350509440/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:461/0 lens 648/0 e 0 to 0 dl 1789302896 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302885.897428:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef31f80 with x1876219350509440 ext(0->157) 00010000:00010000:0.0:1789302885.897439:0:18429:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 11 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.897479:0:18429:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 11 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302885.897489:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ef31f80 00000100:00100000:0.0:1789302885.897705:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 13 00000100:00100000:0.0:1789302885.897717:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350509440:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:0.0:1789302885.898384:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350509440:12345-192.168.203.11@tcp:4:cp.0 Request processed in 666us (1179us total) trans 4294967380 rc 0/0 00000100:00100000:0.0:1789302885.898395:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 13 00002000:00100000:0.0:1789302885.898405:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1d:0x0] (0->157) req@ffff960f3ef31f80 x1876219350509440/t4294967380(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:461/0 lens 648/448 e 0 to 0 dl 1789302896 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302885.898417:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef31f80 with x1876219350509440 ext(0->157) 00010000:00010000:0.0:1789302885.898426:0:18429:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 13 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.898443:0:18429:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 13 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302885.898451:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ef31f80 00000100:00100000:3.0:1789302885.901687:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302885.902042:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350509568 00000100:00100000:0.0:1789302885.902057:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350509568 00010000:00010000:0.0:1789302885.902124:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 13 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302885.902178:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 431 00000100:00100000:0.0:1789302885.902191:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350509568:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302885.902210:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302885.902239:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 13 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.902256:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 13 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.902300:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006bdfc70e/0x64fe43095184f5f6 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e08 expref: 13 pid: 6695 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.902324:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302885.902329:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302885.902337:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.902349:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.902383:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302885.902456:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350509568:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 268us (784us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302885.902476:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 431 00010000:00010000:3.0:1789302885.904415:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904437:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904573:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904585:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904595:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904604:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302885.904610:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904620:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904643:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302885.904651:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe43095184f64a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302885.904772:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350509184:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 25512us (29026us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302885.904783:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 410 00000100:00100000:3.0:1789302885.913690:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302885.913980:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350509824 00000100:00100000:0.0:1789302885.913993:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350509824 00000100:00100000:0.0:1789302885.914064:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1441 00000100:00100000:0.0:1789302885.914075:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32774:x1876219350509824:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302885.914232:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32774:x1876219350509824:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 159us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302885.914240:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1441 00000100:00100000:0.0:1789302885.919092:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302885.920192:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350509952 00000100:00100000:2.0:1789302885.920209:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350509952 00000100:00100000:2.0:1789302885.920280:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 411 00000100:00100000:2.0:1789302885.920291:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350509952:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302885.920323:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302885.920351:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e32 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302885.920383:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1d:0x0] initial LVB size: 158, mtime: 0x6aa69864, atime: 0x0, ctime: 0x6aa69864, blocks: 0x1 00010000:00010000:2.0:1789302885.920415:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=722s v=(0 0) 00010000:00010000:2.0:1789302885.920420:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302885.920431:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.920450:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 12 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.922639:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:2.0:1789302885.922863:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350509952:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2567us (3781us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302885.922875:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 411 00000100:00100000:2.0:1789302885.929643:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302885.929698:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350510208 00000100:00100000:2.0:1789302885.929711:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350510208 00000100:00100000:2.0:1789302885.929783:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1442 00000100:00100000:2.0:1789302885.929796:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22332850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32774:x1876219350510208:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302885.929920:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22332850 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32774:x1876219350510208:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 126us (291us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302885.929928:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1442 00000100:00100000:2.0:1789302885.936998:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302885.943207:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350510336 00000100:00100000:3.0:1789302885.943230:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350510336 00002000:00100000:3.0:1789302885.943304:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1d:0x0] (0->4095) req@ffff960f3f046300 x1876219350510336/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:461/0 lens 488/0 e 0 to 0 dl 1789302896 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302885.943316:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f046300 with x1876219350510336 ext(0->4095) 00002000:00010000:3.0:1789302885.943324:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f046300 00000100:00100000:3.0:1789302885.943356:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 125 00000100:00100000:3.0:1789302885.943365:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350510336:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302885.944825:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350510336:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 1451us (7834us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302885.944838:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 125 00002000:00100000:3.0:1789302885.944848:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x1d:0x0] (0->4095) req@ffff960f3f046300 x1876219350510336/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:461/0 lens 488/4536 e 0 to 0 dl 1789302896 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302885.944858:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f046300 with x1876219350510336 ext(0->4095) 00002000:00010000:3.0:1789302885.944864:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3f046300 00000100:00100000:3.0:1789302885.956024:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.956100:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350510464 00000100:00100000:1.0:1789302885.956127:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350510464 00000100:00100000:1.0:1789302885.956218:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1443 00000100:00100000:1.0:1789302885.956231:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32774:x1876219350510464:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302885.956270:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302885.956299:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f658 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e39 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.956345:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.956362:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.956377:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.956714:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f658 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 56 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.956734:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302885.956745:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:1.0:1789302885.956749:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302885.956754:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc 0) 00000100:00100000:1.0:1789302885.956842:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32774:x1876219350510464:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 613us (828us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302885.956852:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1443 00000100:00100000:2.0:1789302885.965590:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302885.965680:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350510592 00000100:00100000:0.0:1789302885.965694:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350510592 00000100:00100000:0.0:1789302885.965757:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1444 00000100:00100000:0.0:1789302885.965768:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32774:x1876219350510592:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302885.965801:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2c:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302885.965932:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32774:x1876219350510592:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 166us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302885.965939:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1444 00000100:00100000:1.0:1789302885.973346:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302885.973427:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350510720 00000100:00100000:2.0:1789302885.973442:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350510720 00000100:00100000:2.0:1789302885.973603:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 412 00000100:00100000:2.0:1789302885.973614:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350510720:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302885.973664:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973682:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302885.973687:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302885.973693:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973707:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973740:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973749:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973757:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973764:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302885.973768:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973777:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973792:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.973800:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000030d463c9/0x64fe43095184f666 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302885.973881:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350510720:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 269us (552us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302885.973888:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 412 00000100:00100000:3.0:1789302885.976663:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302885.976772:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350510976 00000100:00100000:0.0:1789302885.976787:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350510976 00000100:00100000:0.0:1789302885.976853:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1445 00000100:00100000:0.0:1789302885.976863:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32774:x1876219350510976:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302885.976936:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.976965:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.976975:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.977010:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.977024:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 55 pid: 5845 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302885.977037:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371175424:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302885.977134:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ab25480 going to sleep for 11 seconds 00000100:00100000:3.0:1789302885.980192:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302885.980413:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350511104 00000100:00100000:2.0:1789302885.980428:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350511104 00010000:00010000:2.0:1789302885.980484:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 56 pid: 5845 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302885.980648:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 432 00000100:00100000:2.0:1789302885.980660:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2348:x1876219350511104:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302885.980676:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302885.980684:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 56 pid: 5845 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.980699:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 56 pid: 5845 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.980721:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302885.980726:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.980736:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.980743:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302885.980768:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302885.980836:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2348:x1876219350511104:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 178us (659us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302885.980845:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 432 00010000:00010000:0.0:1789302885.980928:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f65f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e39 expref: 55 pid: 5845 timeout: 822 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302885.980959:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371175424:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302885.980988:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.980999:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.981006:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.985182:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.985206:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.985215:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.985222:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302885.985228:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.985236:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.985268:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:0.0:1789302885.985272:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302885.985278:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302885.985285:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f66d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302885.985612:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32774:x1876219350510976:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 8742us (8953us total) trans 4294968255 rc 0/0 00000100:00100000:0.0:1789302885.985629:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1445 00000040:00080000:1.0:1789302885.985810:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302885.985817:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302885.986310:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:1.0:1789302885.986322:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302885.988375:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302885.988470:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350511360 00000100:00100000:1.0:1789302885.988483:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350511360 00000100:00100000:1.0:1789302885.988589:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 290 00000100:00100000:1.0:1789302885.988602:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32774:x1876219350511360:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302885.989105:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32774:x1876219350511360:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 504us (741us total) trans 4294968256 rc 0/0 00000100:00100000:1.0:1789302885.989132:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 290 00010000:00010000:2.0:1789302886.029668:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302886.100666:0:22483:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302886.100675:0:22483:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302886.100717:0:22483:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 44 count 3 00000040:00080000:1.0:1789302886.100725:0:22483:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302886.159780:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.159856:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350511488 00000100:00100000:1.0:1789302886.159866:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350511488 00000100:00080000:2.0:1789302886.159902:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302886.159938:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 413 00000100:00100000:1.0:1789302886.159950:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350511488:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:1.0:1789302886.159968:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.159986:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e40 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.160012:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302886.160017:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302886.160027:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.160044:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.160057:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8200, rc 0) 00000100:00100000:1.0:1789302886.160129:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859a050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350511488:12345-192.168.203.11@tcp:101:rm.0 Request processed in 179us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302886.160137:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 413 00000100:00100000:1.0:1789302886.173623:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.173677:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350511744 00000100:00100000:2.0:1789302886.173688:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350511744 00000100:00080000:1.0:1789302886.173735:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302886.173762:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1446 00000100:00100000:2.0:1789302886.173776:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:32775:x1876219350511744:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:2.0:1789302886.173827:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1789302886.173853:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f612 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e0f expref: 108 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.173871:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302886.173882:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f627 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e1d expref: 107 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.173891:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.173895:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302886.173917:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.173932:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.173941:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.173961:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.173968:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.173976:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.174074:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.174085:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 55 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.174094:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.174128:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 55 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.174143:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 55 pid: 5844 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302886.174158:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f045180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371175552:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302886.174275:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3358a540 going to sleep for 11 seconds 00000100:00100000:2.0:1789302886.187664:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.187712:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350511872 00000100:00100000:2.0:1789302886.187722:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350511872 00010000:00010000:2.0:1789302886.187771:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 56 pid: 5844 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302886.187806:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00100000:2.0:1789302886.187816:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2348:x1876219350511872:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302886.187830:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302886.187838:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 56 pid: 5844 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.187851:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 56 pid: 5844 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.187871:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302886.187876:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.187884:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.187892:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.187911:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302886.187959:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2348:x1876219350511872:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 144us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302886.187966:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00080000:2.0:1789302886.187987:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302886.188181:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184f5e1 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2c:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dfa expref: 55 pid: 5844 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.188199:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f045180 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371175552:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302886.188217:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.188226:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.188232:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302886.188864:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].15, 64 off9224 00000004:00080000:1.0:1789302886.188875:0:18427:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 723199371640ns, next at 724199369320ns, rc = 0 00010000:00010000:1.0:1789302886.188944:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.188955:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.188964:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.188970:0:18427:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.188976:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.188982:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189002:0:18427:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302886.189019:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189025:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189031:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189037:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.189041:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189048:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189057:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:1.0:1789302886.189060:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302886.189063:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189069:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f689 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2c:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189085:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189091:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189100:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189105:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.189107:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189114:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189123:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.189125:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.189128:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.189133:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095184f67b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302886.189144:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000076b23f11 reply state 00000000f6b4605e transno 4294968257 ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.189153:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302886.189220:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:32775:x1876219350511744:12345-192.168.203.11@tcp:36:rm.0 Request processed in 15448us (15606us total) trans 4294968257 rc 0/0 00000100:00100000:1.0:1789302886.189228:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1446 00010000:00010000:2.0:1789302886.189382:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302886.189389:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.189395:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302886.189399:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.189404:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.189413:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f10d3cc00) 00010000:00010000:2.0:1789302886.189415:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789302886.189418:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.189424:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f690 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2c:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.189439:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302886.189440:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302886.189442:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302886.195095:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.195121:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.195135:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.195142:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.195146:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.195154:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.195169:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f11d36300) 00010000:00010000:1.0:1789302886.195171:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.195176:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.195183:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f682 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302886.389685:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.389782:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350512000 00000100:00100000:3.0:1789302886.389795:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350512000 00000100:00100000:3.0:1789302886.389856:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1447 00000100:00100000:3.0:1789302886.389868:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32791:x1876219350512000:12345-192.168.203.11@tcp:101:mkdir.0 00010000:00010000:3.0:1789302886.389914:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302886.389942:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f697 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e47 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.390028:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f69e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.390046:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f69e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.390061:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f69e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302886.390105:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x51:0x0]: returning size 16384 00000004:00200000:3.0:1789302886.390110:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302886.390135:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f69e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.390166:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f697 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e47 expref: 109 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.390186:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f69e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e47 expref: 108 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.390196:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302886.390199:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302886.390203:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302886.390289:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:32791:x1876219350512000:12345-192.168.203.11@tcp:101:mkdir.0 Request processed in 422us (621us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302886.390297:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1447 00000100:00100000:2.0:1789302886.488448:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.488635:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350512128 00000100:00100000:0.0:1789302886.488646:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350512128 00000100:00100000:0.0:1789302886.488707:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1448 00000100:00100000:0.0:1789302886.488719:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32797:x1876219350512128:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302886.488744:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.488767:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f6a5 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2c:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e4e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.488918:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f6a5 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e4e expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.488932:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2c:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.488935:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.488941:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f6a5 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2c:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e4e expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.488963:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345be00, rc 0) 00000100:00100000:0.0:1789302886.489029:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32797:x1876219350512128:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 312us (591us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302886.489036:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1448 00000100:00100000:2.0:1789302886.495229:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.495668:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350512384 00000100:00100000:1.0:1789302886.495678:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350512384 00000100:00100000:1.0:1789302886.495740:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1449 00000100:00100000:1.0:1789302886.495751:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:32797:x1876219350512384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302886.495770:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.495773:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 1 00010000:00010000:1.0:1789302886.495797:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184f69e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e47 expref: 108 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.495816:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.495821:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302886.495834:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f6ac lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e55 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.496208:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.496222:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.496231:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.496245:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.496253:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.496260:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497314:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497326:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497335:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.497344:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 0 00010000:00010000:1.0:1789302886.497364:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497371:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497377:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497381:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.497384:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497391:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497400:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.497402:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.497405:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497411:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6b3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302886.497422:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000d470b036 reply state 000000004f7c3bd2 transno 4294968258 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.497429:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302886.497469:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184f6ac lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 110 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497487:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 109 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.497525:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.497529:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302886.497532:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9000, rc 0) 00000100:00100000:1.0:1789302886.497678:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:32797:x1876219350512384:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1928us (2457us total) trans 4294968258 rc 0/0 00000100:00100000:1.0:1789302886.497687:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1449 00000100:00100000:1.0:1789302886.503544:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302886.503651:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.503668:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.503676:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.503682:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302886.503687:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.503695:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.503712:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d200) 00010000:00010000:3.0:1789302886.503716:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302886.503721:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.503727:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f6ba lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302886.503921:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350512640 00000100:00100000:0.0:1789302886.503932:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350512640 00000100:00100000:0.0:1789302886.503986:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1450 00000100:00100000:0.0:1789302886.503996:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:32797:x1876219350512640:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302886.504019:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.504035:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f6cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e5c expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.504130:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.504144:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 110 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.504152:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.504198:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 110 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.504213:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 110 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302886.504226:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371175680:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302886.504315:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302886.513451:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.513595:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63ca80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371175680:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302886.513623:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302886.522832:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350512768 00000100:00100000:2.0:1789302886.522847:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350512768 00010000:00010000:2.0:1789302886.522900:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 111 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302886.522951:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 434 00000100:00100000:2.0:1789302886.522962:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:2348:x1876219350512768:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302886.522977:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302886.522985:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 111 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.522999:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 111 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.523019:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe43095184f6c1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e55 expref: 111 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.523038:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:2.0:1789302886.523043:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.523051:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.523059:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.523097:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302886.523160:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2348:x1876219350512768:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 200us (9718us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302886.523168:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 434 00010000:00010000:1.0:1789302886.523479:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523492:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.523540:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 0 00010000:00010000:1.0:1789302886.523799:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523807:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523813:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523819:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.523822:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523829:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523840:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.523842:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302886.523846:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523852:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f6d6 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523910:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f6cf lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e5c expref: 110 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523921:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.523924:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.523927:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f6cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e5c expref: 110 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.523937:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345be00, rc 0) 00000100:00100000:1.0:1789302886.523979:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:32797:x1876219350512640:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 19985us (20482us total) trans 4294968259 rc 301/301 00000100:00100000:1.0:1789302886.523987:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1450 00000100:00100000:1.0:1789302886.527116:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.527211:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350513024 00000100:00100000:0.0:1789302886.527221:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350513024 00000100:00100000:0.0:1789302886.527290:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 291 00000100:00100000:0.0:1789302886.527301:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:32797:x1876219350513024:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302886.527724:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:32797:x1876219350513024:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 423us (624us total) trans 4294968260 rc 0/0 00000100:00100000:0.0:1789302886.527732:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 291 00000100:00100000:2.0:1789302886.535311:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.535814:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350513152 00000100:00100000:0.0:1789302886.535825:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350513152 00000100:00100000:0.0:1789302886.535887:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1451 00000100:00100000:0.0:1789302886.535897:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:32797:x1876219350513152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302886.535922:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.535943:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f6e4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e63 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.535999:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f6e4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e63 expref: 111 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.536013:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f6e4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e63 expref: 111 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.536033:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.536037:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302886.536041:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b600, rc 0) 00000100:00100000:0.0:1789302886.537994:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:32797:x1876219350513152:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 2087us (2679us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302886.538006:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1451 00000100:00100000:2.0:1789302886.541616:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.545741:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350513408 00000100:00100000:1.0:1789302886.545757:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350513408 00000100:00100000:1.0:1789302886.545841:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1452 00000100:00100000:1.0:1789302886.545852:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:32797:x1876219350513408:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1789302886.545922:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 3/0,1 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.545945:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.545957:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.546190:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.546199:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.546205:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.546212:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.546216:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.546224:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.546236:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.546240:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1789302886.546245:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 2/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.546251:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f6eb lrc: 0/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.546342:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:32797:x1876219350513408:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 491us (4734us total) trans 4294968261 rc 0/0 00000100:00100000:1.0:1789302886.546351:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1452 00000100:00100000:1.0:1789302886.549802:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.549875:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350513664 00000100:00100000:0.0:1789302886.549885:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350513664 00000100:00100000:0.0:1789302886.549943:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1453 00000100:00100000:0.0:1789302886.549952:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:32797:x1876219350513664:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302886.550221:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.550237:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.550248:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302886.550320:0:5843:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968262 00000004:00010000:0.0:1789302886.550362:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000786098f2 reply state 000000002d844908 transno 4294968263 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302886.550371:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302886.550781:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:32797:x1876219350513664:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 828us (986us total) trans 4294968263 rc 0/0 00000100:00100000:0.0:1789302886.550788:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1453 00000020:00080000:0.0:1789302886.550793:0:5843:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:1.0:1789302886.557944:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.557959:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.557967:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.557973:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.557978:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.557985:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.558000:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.558003:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302886.558008:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.558014:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f6f2 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.559477:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.559987:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350513920 00000100:00100000:3.0:1789302886.560000:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350513920 00000100:00100000:3.0:1789302886.560059:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00100000:3.0:1789302886.560069:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:32797:x1876219350513920:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302886.560318:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:32797:x1876219350513920:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 249us (847us total) trans 4294968264 rc 0/0 00000100:00100000:3.0:1789302886.560326:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 292 00000100:00100000:2.0:1789302886.607686:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.607791:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350514176 00000100:00100000:0.0:1789302886.607801:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350514176 00000100:00100000:0.0:1789302886.607863:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1454 00000100:00100000:0.0:1789302886.607873:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:32800:x1876219350514176:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302886.607898:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.607920:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f6f9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e6a expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.607992:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608006:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608017:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608031:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608038:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608044:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608166:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f70e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608174:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f70e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608181:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f70e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302886.608189:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1789302886.608280:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608293:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608300:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608306:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.608310:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608318:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608329:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.608332:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.608336:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608343:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f700 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608359:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608365:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608372:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608376:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.608379:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608386:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608392:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f09ae1f00) 00010000:00010000:0.0:1789302886.608395:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.608398:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608404:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f707 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608443:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f6f9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e6a expref: 118 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608472:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f70e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e6a expref: 117 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.608480:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.608483:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302886.608485:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ae00, rc 0) 00000100:00100000:0.0:1789302886.608615:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:32800:x1876219350514176:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 742us (937us total) trans 4294968265 rc 0/0 00000100:00100000:0.0:1789302886.608623:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1454 00000100:00100000:2.0:1789302886.615604:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.615781:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350514304 00000100:00100000:0.0:1789302886.615792:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350514304 00000100:00100000:0.0:1789302886.615850:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1455 00000100:00100000:0.0:1789302886.615861:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:32800:x1876219350514304:12345-192.168.203.11@tcp:54:lfs.0 00010000:00010000:0.0:1789302886.615926:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.615942:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.615953:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.615977:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.615983:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.615989:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.615995:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.615999:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.616007:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.616017:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.616020:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302886.616024:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.616030:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f71c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302886.616112:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:32800:x1876219350514304:12345-192.168.203.11@tcp:54:lfs.0 Request processed in 253us (517us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302886.616119:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1455 00000100:00100000:2.0:1789302886.618916:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.619008:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350514560 00000100:00100000:1.0:1789302886.619017:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350514560 00000100:00100000:1.0:1789302886.619070:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:1.0:1789302886.619080:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076eb450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:32800:x1876219350514560:12345-192.168.203.11@tcp:35:lfs.0 00000100:00100000:1.0:1789302886.619443:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076eb450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:32800:x1876219350514560:12345-192.168.203.11@tcp:35:lfs.0 Request processed in 365us (535us total) trans 4294968266 rc 0/0 00000100:00100000:1.0:1789302886.619449:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 293 00000100:00100000:3.0:1789302886.692692:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.692852:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350514816 00000100:00100000:0.0:1789302886.692864:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350514816 00000100:00100000:0.0:1789302886.692934:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1456 00000100:00100000:0.0:1789302886.692946:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:32806:x1876219350514816:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:0.0:1789302886.692968:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.692989:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f723 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e71 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.693092:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f72a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.693114:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f72a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.693126:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f72a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302886.693136:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1789302886.693255:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f723 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e71 expref: 121 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.693276:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f72a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e71 expref: 120 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.693284:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.693287:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:0.0:1789302886.693292:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345aa00, rc 0) 00000100:00100000:0.0:1789302886.693350:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+120:32806:x1876219350514816:12345-192.168.203.11@tcp:101:cat.0 Request processed in 406us (667us total) trans 4294968267 rc 0/0 00000100:00100000:0.0:1789302886.693357:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1456 00000100:00100000:3.0:1789302886.699604:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.700154:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350515072 00000100:00100000:1.0:1789302886.700175:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350515072 00000100:00100000:1.0:1789302886.700239:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1457 00000100:00100000:1.0:1789302886.700250:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+120:32805:x1876219350515072:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.700272:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.700292:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f738 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e78 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.700389:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.700405:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.700416:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.700427:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.700621:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095184f738 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 123 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.700652:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 122 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.700661:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.700665:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint d 00010000:00010000:1.0:1789302886.700668:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8e00, rc 0) 00000100:00100000:1.0:1789302886.700741:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32805:x1876219350515072:12345-192.168.203.11@tcp:101:cat.0 Request processed in 492us (1146us total) trans 4294968268 rc 0/0 00000100:00100000:1.0:1789302886.700748:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1457 00000100:00100000:3.0:1789302886.714366:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.715073:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350515840 00000100:00100000:2.0:1789302886.715084:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350515840 00000100:00100000:2.0:1789302886.715150:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1458 00000100:00100000:2.0:1789302886.715163:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32807:x1876219350515840:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302886.715188:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302886.715209:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f74d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e7f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715283:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715298:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715309:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715323:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715330:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715336:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715464:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715473:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715481:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302886.715490:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1789302886.715637:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715645:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715652:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715657:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.715662:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715669:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715680:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.715683:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.715688:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715694:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f754 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715720:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715727:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715736:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715741:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.715744:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715750:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715758:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f11d36300) 00010000:00010000:2.0:1789302886.715760:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.715763:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715769:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f75b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715808:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f74d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 125 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715824:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 124 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.715832:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302886.715834:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302886.715836:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030800, rc 0) 00000100:00100000:2.0:1789302886.715904:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+124:32807:x1876219350515840:12345-192.168.203.11@tcp:101:cat.0 Request processed in 744us (1546us total) trans 4294968269 rc 0/0 00000100:00100000:2.0:1789302886.715914:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1458 00000100:00100000:3.0:1789302886.719074:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.719123:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.719153:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350515584 00000100:00100000:0.0:1789302886.719165:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350515584 00000100:00100000:2.0:1789302886.719341:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350515456 00000100:00100000:2.0:1789302886.719346:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350515456 00000100:00100000:2.0:1789302886.719389:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 68 00000100:00100000:2.0:1789302886.719399:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd05450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:32805:x1876219350515456:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:2.0:1789302886.719415:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1789302886.719439:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f70e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e6a expref: 125 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.719476:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f72a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e71 expref: 124 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.719488:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f6e4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e63 expref: 123 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.719499:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302886.719527:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302886.719579:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd05450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32805:x1876219350515456:12345-192.168.203.11@tcp:103:cat.0 Request processed in 180us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302886.719587:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 68 00000100:00100000:0.0:1789302886.720202:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 294 00000100:00100000:0.0:1789302886.720215:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+121:32806:x1876219350515584:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:0.0:1789302886.720430:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32806:x1876219350515584:12345-192.168.203.11@tcp:35:cat.0 Request processed in 219us (1365us total) trans 4294968270 rc 0/0 00000100:00100000:0.0:1789302886.720438:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 294 00000100:00100000:2.0:1789302886.728525:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.728727:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350515968 00000100:00100000:1.0:1789302886.728738:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350515968 00000100:00100000:1.0:1789302886.728798:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1459 00000100:00100000:1.0:1789302886.728808:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32805:x1876219350515968:12345-192.168.203.11@tcp:58:cat.0 00010000:00010000:1.0:1789302886.728900:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.728919:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 122 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.728928:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 122 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.728936:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 5/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.728975:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 122 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.728997:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 122 pid: 5845 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.729010:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371175808:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302886.729104:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 122 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.729114:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 122 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.729122:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ad880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371175936:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302886.729143:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33589cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302886.731076:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.731425:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350516224 00000100:00100000:3.0:1789302886.731434:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350516224 00000100:00100000:3.0:1789302886.731525:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 295 00000100:00100000:3.0:1789302886.731537:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+123:32807:x1876219350516224:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302886.731972:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+124:32807:x1876219350516224:12345-192.168.203.11@tcp:35:cat.0 Request processed in 436us (901us total) trans 4294968271 rc 0/0 00000100:00100000:3.0:1789302886.731979:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 295 00000100:00100000:1.0:1789302886.743860:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ae300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371175808:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302886.743877:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ad880 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371175936:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302886.743906:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302886.748793:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.748833:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.748870:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.748906:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.748969:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350516608 00000100:00100000:0.0:1789302886.748981:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350516608 00000100:00100000:0.0:1789302886.749039:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350516736 00000100:00100000:0.0:1789302886.749043:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350516736 00000100:00100000:0.0:1789302886.749071:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1460 00000100:00100000:0.0:1789302886.749083:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:32812:x1876219350516608:12345-192.168.203.11@tcp:101:cat.0 00000100:00100000:1.0:1789302886.749091:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1461 00000100:00100000:1.0:1789302886.749103:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:32813:x1876219350516736:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:0.0:1789302886.749107:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.749120:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.749130:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f777 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e86 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.749136:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f77e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 7 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e8d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.749220:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f785 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 8 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.749222:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.749245:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f785 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.749257:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f785 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302886.749284:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00000004:00200000:0.0:1789302886.749288:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000100:00100000:2.0:1789302886.749300:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350516352 00000100:00100000:2.0:1789302886.749305:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350516352 00000004:00010000:0.0:1789302886.749307:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f785 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.749325:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 128 pid: 5845 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302886.749349:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350516480 00000100:00100000:2.0:1789302886.749353:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350516480 00010000:00010000:2.0:1789302886.749365:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 8 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 129 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302886.749384:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 435 00000100:00100000:2.0:1789302886.749391:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:2347:x1876219350516352:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302886.749401:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302886.749407:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 9 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 129 pid: 5845 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302886.749421:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 436 00000100:00100000:3.0:1789302886.749432:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:2347:x1876219350516480:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302886.749442:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302886.749448:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 10 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 129 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.750293:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.750309:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.750339:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00000004:00200000:1.0:1789302886.750343:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302886.750357:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 10 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.750449:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f77e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 10 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 130 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.750552:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 129 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.750560:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.750563:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.750567:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9600, rc 0) 00000100:00100000:1.0:1789302886.750638:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:32813:x1876219350516736:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1537us (1730us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302886.750656:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1461 00010000:00010000:0.0:1789302886.751142:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f777 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 9 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e86 expref: 129 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.751160:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f785 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 8 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e86 expref: 128 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.751167:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.751170:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302886.751172:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302886.751202:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+128:32812:x1876219350516608:12345-192.168.203.11@tcp:101:cat.0 Request processed in 2122us (2331us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302886.751208:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1460 00000100:00100000:1.0:1789302886.752387:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.752870:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350516992 00000100:00100000:0.0:1789302886.752880:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350516992 00000100:00100000:0.0:1789302886.752936:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1462 00000100:00100000:0.0:1789302886.752948:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+128:32814:x1876219350516992:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:0.0:1789302886.752968:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.752988:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f793 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e94 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753058:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753075:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753086:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753101:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753107:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753113:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753234:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 9 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753244:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753252:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302886.753262:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1789302886.753360:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753369:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753375:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753381:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.753386:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753393:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753404:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.753407:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302886.753412:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753417:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f7a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 9 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753434:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753440:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753445:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753462:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.753465:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753471:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753483:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.753486:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.753489:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753494:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f79a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753525:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753532:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753538:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753543:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ae00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.753546:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753551:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753558:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f09ae1f00) 00010000:00010000:0.0:1789302886.753560:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.753563:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753568:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f7a1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753612:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f793 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e94 expref: 130 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753621:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.753623:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302886.753627:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184f793 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e94 expref: 130 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.753636:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b600, rc 0) 00000100:00100000:0.0:1789302886.753696:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:32814:x1876219350516992:12345-192.168.203.11@tcp:101:cat.0 Request processed in 750us (1313us total) trans 4294968272 rc 301/301 00000100:00100000:0.0:1789302886.753703:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1462 00010000:00010000:2.0:1789302886.756596:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 128 pid: 5845 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.756629:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f762 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x1b/0x0 rrc: 8 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e7f expref: 128 pid: 5845 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.756647:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302886.756652:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 7 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.756661:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302886.756732:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+127:2347:x1876219350516352:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 7340us (7948us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302886.756742:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 435 00010000:00010000:3.0:1789302886.757825:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 126 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.757922:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095184f73f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0xd/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e78 expref: 126 pid: 18427 timeout: 823 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.757942:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302886.757947:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.758187:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.758198:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.758284:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302886.758342:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:2347:x1876219350516480:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 8912us (9507us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302886.758351:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 436 00010000:00010000:1.0:1789302886.758644:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.758654:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302886.759129:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].44, 136 off14176 00000100:00100000:1.0:1789302886.759204:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+124:32805:x1876219350515968:12345-192.168.203.11@tcp:58:cat.0 Request processed in 30397us (30703us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302886.759214:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1459 00000100:00100000:2.0:1789302886.768781:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.770625:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350517248 00000100:00100000:1.0:1789302886.770641:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350517248 00000100:00100000:1.0:1789302886.770822:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 296 00000100:00100000:1.0:1789302886.770847:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+124:32814:x1876219350517248:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302886.771102:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:32814:x1876219350517248:12345-192.168.203.11@tcp:35:cat.0 Request processed in 268us (2329us total) trans 4294968273 rc 0/0 00000100:00100000:1.0:1789302886.776274:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 296 00000100:00100000:0.0:1789302886.776668:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.777347:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350517504 00000100:00100000:1.0:1789302886.777358:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350517504 00000100:00100000:1.0:1789302886.777536:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1463 00000100:00100000:1.0:1789302886.777547:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:32810:x1876219350517504:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.777565:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.777584:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7b6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0e9b expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777652:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777665:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777676:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777690:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777707:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777715:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777821:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.777841:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778033:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.778046:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.778141:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778149:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778155:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778161:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.778165:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778177:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778189:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.778192:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.778196:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778202:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7cb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778217:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778223:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778229:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778234:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.778236:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778242:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778248:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.778251:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.778254:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778262:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778271:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778277:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778283:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778288:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.778291:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778297:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778303:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.778305:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.778308:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778313:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778352:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7b6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e9b expref: 127 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778442:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.778445:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.778449:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7b6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e9b expref: 127 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.778469:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9400, rc 0) 00000100:00100000:1.0:1789302886.778582:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+126:32810:x1876219350517504:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1037us (1915us total) trans 4294968274 rc 301/301 00000100:00100000:1.0:1789302886.778589:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1463 00000100:00100000:2.0:1789302886.783873:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.783967:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.783969:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350517632 00000100:00100000:3.0:1789302886.783981:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350517632 00000100:00100000:3.0:1789302886.784057:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 69 00000100:00100000:2.0:1789302886.784924:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+126:2348:x1876219350517632:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302886.784944:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302886.784968:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184f785 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e86 expref: 126 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.784988:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302886.784992:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302886.785053:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:2348:x1876219350517632:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 984us (1188us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302886.785061:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 69 00000100:00100000:1.0:1789302886.789835:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350517760 00000100:00100000:1.0:1789302886.789852:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350517760 00000100:00100000:1.0:1789302886.789925:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1464 00000100:00100000:1.0:1789302886.789935:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:32805:x1876219350517760:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.789959:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.789980:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ea2 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.790035:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 2/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.790045:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 4/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 126 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.790062:0:6919:0:(ldlm_request.c:385:ldlm_blocking_ast_nocheck()) ### Lock still has references, will be cancelled later ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.790078:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 126 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.790091:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.790094:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1789302886.790098:0:6919:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 3/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 126 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.790106:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9400, rc 0) 00000100:00100000:1.0:1789302886.790175:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+126:32805:x1876219350517760:12345-192.168.203.11@tcp:101:cat.0 Request processed in 241us (6207us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302886.790182:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1464 00000100:00100000:2.0:1789302886.794390:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.794572:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350518016 00000100:00100000:0.0:1789302886.794582:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350518016 00000100:00100000:0.0:1789302886.794637:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 297 00000100:00100000:0.0:1789302886.794648:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+126:32810:x1876219350518016:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:0.0:1789302886.794875:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+127:32810:x1876219350518016:12345-192.168.203.11@tcp:35:cat.0 Request processed in 228us (493us total) trans 4294968275 rc 0/0 00000100:00100000:0.0:1789302886.794882:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 297 00000100:00100000:2.0:1789302886.795914:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.795975:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350518272 00000100:00100000:1.0:1789302886.795983:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350518272 00000100:00100000:1.0:1789302886.796035:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1465 00000100:00100000:1.0:1789302886.796043:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+127:32811:x1876219350518272:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.796058:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.796078:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7e0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ea9 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796144:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796157:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796167:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796182:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796189:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796195:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796305:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796315:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796322:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.796332:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.796406:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796414:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796421:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796427:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.796431:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796443:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796470:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.796474:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.796479:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796485:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f7f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796639:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796648:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796654:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796659:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.796662:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796668:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796676:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.796679:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.796683:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796692:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f7e7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796702:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796708:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796788:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796795:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.796798:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796805:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796813:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.796816:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.796819:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796824:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f7ee lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796867:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7e0 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea9 expref: 129 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796882:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.796884:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.796888:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f7e0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea9 expref: 129 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.796896:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302886.797002:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+128:32811:x1876219350518272:12345-192.168.203.11@tcp:101:cat.0 Request processed in 958us (1090us total) trans 4294968276 rc 301/301 00000100:00100000:1.0:1789302886.797009:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1465 00000100:00100000:3.0:1789302886.805642:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.805676:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.805861:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350518528 00000100:00100000:1.0:1789302886.805870:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350518528 00000100:00100000:1.0:1789302886.805922:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 298 00000100:00100000:1.0:1789302886.805932:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+128:32811:x1876219350518528:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302886.806161:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:32811:x1876219350518528:12345-192.168.203.11@tcp:35:cat.0 Request processed in 230us (527us total) trans 4294968277 rc 0/0 00000100:00100000:1.0:1789302886.806169:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 298 00000100:00100000:0.0:1789302886.808696:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350518784 00000100:00100000:0.0:1789302886.808709:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350518784 00000100:00100000:0.0:1789302886.809885:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1466 00000100:00100000:0.0:1789302886.809896:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:32808:x1876219350518784:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:0.0:1789302886.809915:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.809936:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f803 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0eb0 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810006:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810020:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810030:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810042:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810049:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810055:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810158:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810166:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810173:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302886.810182:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1789302886.810257:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810269:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810275:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810281:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.810285:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810292:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810303:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.810306:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302886.810310:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810316:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f818 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810331:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810337:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810343:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810347:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.810351:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810357:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810363:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.810365:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.810368:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810373:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f80a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810381:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810386:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810391:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810396:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.810399:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810405:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810411:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f09ae1f00) 00010000:00010000:0.0:1789302886.810413:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.810416:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810421:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f811 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810466:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f803 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eb0 expref: 131 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810475:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.810477:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302886.810480:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f803 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eb0 expref: 131 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.810490:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ae00, rc 0) 00000100:00100000:0.0:1789302886.810575:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+130:32808:x1876219350518784:12345-192.168.203.11@tcp:101:cat.0 Request processed in 680us (4900us total) trans 4294968278 rc 301/301 00000100:00100000:0.0:1789302886.810582:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1466 00000100:00100000:2.0:1789302886.817568:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.817781:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350519040 00000100:00100000:0.0:1789302886.817792:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350519040 00000100:00100000:0.0:1789302886.817855:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 299 00000100:00100000:0.0:1789302886.817865:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ebc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+130:32808:x1876219350519040:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:0.0:1789302886.818103:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ebc50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+131:32808:x1876219350519040:12345-192.168.203.11@tcp:35:cat.0 Request processed in 239us (543us total) trans 4294968279 rc 0/0 00000100:00100000:0.0:1789302886.818110:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 299 00000100:00100000:3.0:1789302886.828057:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.828139:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350519296 00000100:00100000:1.0:1789302886.828149:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350519296 00000100:00100000:1.0:1789302886.828216:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1467 00000100:00100000:1.0:1789302886.828227:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+131:32816:x1876219350519296:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.828247:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.828267:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f826 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0eb7 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828340:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828355:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828366:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828381:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828389:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828396:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828570:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828580:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828588:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.828598:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.828689:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828698:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828704:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828711:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.828896:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828905:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828916:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.828919:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.828924:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828930:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f83b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828946:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828953:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828958:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828963:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.828966:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828972:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828983:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.828985:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.828988:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.828994:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f82d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829003:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829010:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829016:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829021:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.829023:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829029:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829036:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.829038:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.829041:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829046:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f834 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829091:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f826 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eb7 expref: 133 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829102:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.829105:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.829108:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f826 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eb7 expref: 133 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.829117:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302886.829235:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+132:32816:x1876219350519296:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1009us (1185us total) trans 4294968280 rc 301/301 00000100:00100000:1.0:1789302886.829243:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1467 00000100:00100000:3.0:1789302886.836951:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.836991:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.838218:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350519808 00000100:00100000:0.0:1789302886.838231:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350519808 00000100:00100000:0.0:1789302886.838302:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1468 00000100:00100000:0.0:1789302886.838313:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+132:32821:x1876219350519808:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:0.0:1789302886.838340:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.838362:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f849 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ebe expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838436:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838461:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838472:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838487:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838495:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838537:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838667:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838676:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838684:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302886.838699:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1789302886.838796:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838804:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838811:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838816:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.838820:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838827:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838838:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.838841:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302886.838845:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838851:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f85e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838869:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838875:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838885:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838889:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.838892:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838898:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838904:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.838907:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.838909:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838915:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f850 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838924:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838930:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838936:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838941:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.838943:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838952:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838959:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f09ae1f00) 00010000:00010000:0.0:1789302886.838961:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.838964:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.838970:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184f857 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.839008:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f849 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ebe expref: 134 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.839018:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.839021:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302886.839024:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184f849 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ebe expref: 134 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.839033:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ae00, rc 0) 00000100:00100000:0.0:1789302886.839092:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+133:32821:x1876219350519808:12345-192.168.203.11@tcp:101:cat.0 Request processed in 781us (2100us total) trans 4294968281 rc 301/301 00000100:00100000:0.0:1789302886.839099:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1468 00000100:00100000:1.0:1789302886.839849:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350519552 00000100:00100000:1.0:1789302886.839859:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350519552 00000100:00100000:1.0:1789302886.839901:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:1.0:1789302886.839910:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+133:32816:x1876219350519552:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302886.840041:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+134:32816:x1876219350519552:12345-192.168.203.11@tcp:35:cat.0 Request processed in 133us (3100us total) trans 4294968282 rc 0/0 00000100:00100000:1.0:1789302886.840048:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 300 00000100:00100000:3.0:1789302886.849105:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.849143:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.849292:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350520320 00000100:00100000:0.0:1789302886.849302:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350520320 00000100:00100000:0.0:1789302886.849345:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:0.0:1789302886.849353:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+134:32821:x1876219350520320:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:0.0:1789302886.849582:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+135:32821:x1876219350520320:12345-192.168.203.11@tcp:35:cat.0 Request processed in 229us (437us total) trans 4294968283 rc 0/0 00000100:00100000:0.0:1789302886.849589:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 301 00000100:00100000:1.0:1789302886.849839:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350520064 00000100:00100000:1.0:1789302886.849844:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350520064 00000100:00100000:1.0:1789302886.849879:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1469 00000100:00100000:1.0:1789302886.849885:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+135:32823:x1876219350520064:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.849895:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.849912:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f86c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ec5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.849968:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.849981:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.849992:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850005:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850012:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850018:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850105:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850114:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850120:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.850129:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.850176:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850188:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850194:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850200:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.850204:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850211:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850221:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.850224:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.850229:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850235:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f881 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850248:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850253:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850259:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850265:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.850268:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850274:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850280:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.850282:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.850284:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850289:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f873 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850296:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850301:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850307:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850312:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.850314:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850320:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850327:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.850329:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.850332:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850337:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f87a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850365:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f86c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ec5 expref: 137 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850374:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.850376:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.850380:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f86c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ec5 expref: 137 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.850387:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302886.850428:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+136:32823:x1876219350520064:12345-192.168.203.11@tcp:101:cat.0 Request processed in 543us (1331us total) trans 4294968284 rc 301/301 00000100:00100000:1.0:1789302886.850434:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1469 00000100:00100000:2.0:1789302886.868208:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.868303:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350520832 00000100:00100000:1.0:1789302886.868313:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350520832 00000100:00100000:1.0:1789302886.868410:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1470 00000100:00100000:1.0:1789302886.868420:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+136:32818:x1876219350520832:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.868443:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.868476:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f88f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ecc expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868589:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868609:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868620:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868651:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868660:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868667:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868946:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868957:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.868964:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.868975:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.869061:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869069:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869076:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869082:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.869087:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869095:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869107:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.869109:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.869115:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869121:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8a4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869136:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869142:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869148:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869153:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.869156:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869161:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869172:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.869174:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.869177:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869182:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f896 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869190:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869196:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869201:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869206:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.869209:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869215:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869222:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.869224:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.869227:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869232:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f89d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869275:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f88f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ecc expref: 138 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869285:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.869288:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.869292:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f88f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ecc expref: 138 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.869301:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302886.869355:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+137:32818:x1876219350520832:12345-192.168.203.11@tcp:101:cat.0 Request processed in 936us (1155us total) trans 4294968285 rc 301/301 00000100:00100000:1.0:1789302886.869362:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1470 00000100:00100000:2.0:1789302886.870575:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.873056:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350520704 00000100:00100000:0.0:1789302886.873073:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350520704 00000100:00100000:0.0:1789302886.873140:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 302 00000100:00100000:0.0:1789302886.873150:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+137:32823:x1876219350520704:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:0.0:1789302886.873375:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+138:32823:x1876219350520704:12345-192.168.203.11@tcp:35:cat.0 Request processed in 226us (2803us total) trans 4294968286 rc 0/0 00000100:00100000:0.0:1789302886.873383:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 302 00000100:00100000:3.0:1789302886.873706:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.873809:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350521088 00000100:00100000:3.0:1789302886.873814:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350521088 00000100:00100000:3.0:1789302886.873850:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1471 00000100:00100000:3.0:1789302886.873857:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+138:32819:x1876219350521088:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302886.873906:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302886.873927:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f8b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ed3 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.873996:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874010:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874021:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874035:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874043:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874049:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874150:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874159:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874167:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302886.874176:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302886.874238:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874250:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874257:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874264:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302886.874268:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874277:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874288:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302886.874290:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302886.874295:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874301:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184f8c7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874314:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874320:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874327:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874332:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302886.874335:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874342:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874349:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302886.874351:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302886.874354:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874361:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184f8b9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874369:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874376:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874382:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874388:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302886.874391:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874397:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874404:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302886.874407:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302886.874409:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874415:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184f8c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874463:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f8b2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ed3 expref: 140 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874474:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302886.874476:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302886.874480:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184f8b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ed3 expref: 140 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302886.874490:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302886.874569:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+139:32819:x1876219350521088:12345-192.168.203.11@tcp:101:cat.0 Request processed in 712us (864us total) trans 4294968287 rc 301/301 00000100:00100000:3.0:1789302886.874577:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1471 00000100:00100000:3.0:1789302886.882002:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.882086:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350521344 00000100:00100000:1.0:1789302886.882097:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350521344 00000100:00100000:1.0:1789302886.882153:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 303 00000100:00100000:1.0:1789302886.882163:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+139:32818:x1876219350521344:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302886.882396:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+140:32818:x1876219350521344:12345-192.168.203.11@tcp:35:cat.0 Request processed in 233us (401us total) trans 4294968288 rc 0/0 00000100:00100000:1.0:1789302886.882403:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 303 00000100:00100000:0.0:1789302886.889654:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.889704:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.890846:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350521856 00000100:00100000:3.0:1789302886.890859:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350521856 00000100:00100000:3.0:1789302886.890929:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 304 00000100:00100000:3.0:1789302886.890941:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+140:32819:x1876219350521856:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302886.891180:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+141:32819:x1876219350521856:12345-192.168.203.11@tcp:35:cat.0 Request processed in 240us (1473us total) trans 4294968289 rc 0/0 00000100:00100000:3.0:1789302886.891187:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 304 00000100:00100000:1.0:1789302886.891403:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350521600 00000100:00100000:1.0:1789302886.891408:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350521600 00000100:00100000:1.0:1789302886.891446:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1472 00000100:00100000:1.0:1789302886.891465:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+141:32820:x1876219350521600:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.891475:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.891494:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f8d5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0eda expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891579:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891593:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891603:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891616:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891623:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891629:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891731:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891740:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891747:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.891756:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.891811:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891823:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891831:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891837:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.891841:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891848:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891859:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.891862:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.891866:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891872:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f8ea lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891886:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891892:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891898:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891903:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.891906:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891912:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891918:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.891920:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.891923:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891928:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f8dc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891935:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891940:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891946:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891951:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.891953:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891959:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891966:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.891968:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.891971:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.891976:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8e3 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.892005:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f8d5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eda expref: 143 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.892015:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.892017:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.892020:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f8d5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eda expref: 143 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.892028:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302886.892068:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+142:32820:x1876219350521600:12345-192.168.203.11@tcp:101:cat.0 Request processed in 615us (2423us total) trans 4294968290 rc 301/301 00000100:00100000:1.0:1789302886.892075:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1472 00000100:00100000:2.0:1789302886.914229:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.914348:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.915662:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350522368 00000100:00100000:1.0:1789302886.915674:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350522368 00000100:00100000:1.0:1789302886.915742:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:1.0:1789302886.915754:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e9450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+142:32820:x1876219350522368:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302886.915986:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e9450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+143:32820:x1876219350522368:12345-192.168.203.11@tcp:35:cat.0 Request processed in 234us (1636us total) trans 4294968291 rc 0/0 00000100:00100000:1.0:1789302886.915993:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 305 00000100:00100000:2.0:1789302886.922369:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350522112 00000100:00100000:2.0:1789302886.922386:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350522112 00000100:00100000:1.0:1789302886.922610:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1473 00000100:00100000:1.0:1789302886.922631:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+143:32829:x1876219350522112:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.922658:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.922681:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f8f8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ee1 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922775:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922790:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922802:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922818:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922827:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922835:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922951:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922961:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.922969:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.922979:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.923072:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923081:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923088:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923095:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.923100:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923107:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923119:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.923122:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.923127:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923133:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f90d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923146:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923152:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923158:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923168:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.923171:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923177:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923183:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.923186:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.923189:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923195:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f8ff lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923204:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923211:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923217:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923222:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.923225:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923231:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923243:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.923245:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.923248:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923253:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f906 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923292:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f8f8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ee1 expref: 145 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923302:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.923305:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.923309:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe43095184f8f8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ee1 expref: 145 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.923318:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302886.923384:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+144:32829:x1876219350522112:12345-192.168.203.11@tcp:101:cat.0 Request processed in 763us (9164us total) trans 4294968292 rc 301/301 00000100:00100000:1.0:1789302886.923393:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1473 00000100:00100000:2.0:1789302886.930165:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.930210:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.930241:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350522880 00000100:00100000:2.0:1789302886.930250:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350522880 00000100:00100000:2.0:1789302886.930301:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1474 00000100:00100000:2.0:1789302886.930311:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+144:32834:x1876219350522880:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302886.930334:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302886.930353:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f91b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ee8 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930424:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930438:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930462:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930477:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930485:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930492:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302886.930494:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350522752 00000100:00100000:3.0:1789302886.930528:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350522752 00000100:00100000:3.0:1789302886.930565:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 306 00000100:00100000:3.0:1789302886.930574:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+146:32829:x1876219350522752:12345-192.168.203.11@tcp:35:cat.0 00010000:00010000:2.0:1789302886.930677:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930687:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.930705:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302886.930716:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00000100:00100000:3.0:1789302886.930913:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+148:32829:x1876219350522752:12345-192.168.203.11@tcp:35:cat.0 Request processed in 339us (755us total) trans 4294968293 rc 0/0 00000100:00100000:3.0:1789302886.930921:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 306 00010000:00010000:2.0:1789302886.934933:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.934952:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.934960:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.934967:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.934972:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.934980:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.934998:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302886.935002:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302886.935007:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935013:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184f930 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935032:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935039:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935045:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935050:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.935054:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935060:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935069:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.935072:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.935075:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935082:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f922 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935094:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935101:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935108:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935113:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.935116:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935121:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935128:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f11d36300) 00010000:00010000:2.0:1789302886.935130:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.935134:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935139:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f929 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935192:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f91b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ee8 expref: 147 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935202:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.935205:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302886.935208:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184f91b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ee8 expref: 147 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.935218:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030200, rc 0) 00000100:00100000:2.0:1789302886.935279:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+146:32834:x1876219350522880:12345-192.168.203.11@tcp:101:cat.0 Request processed in 4968us (5066us total) trans 4294968294 rc 301/301 00000100:00100000:2.0:1789302886.935288:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1474 00000100:00100000:2.0:1789302886.943087:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302886.943163:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.943224:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350523392 00000100:00100000:2.0:1789302886.943232:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350523136 00000100:00100000:3.0:1789302886.943236:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350523392 00000100:00100000:2.0:1789302886.943240:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350523136 00000100:00100000:2.0:1789302886.943317:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1475 00000100:00100000:2.0:1789302886.943341:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+147:32828:x1876219350523136:12345-192.168.203.11@tcp:101:cat.0 00000100:00100000:3.0:1789302886.943374:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 307 00010000:00010000:2.0:1789302886.943374:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1789302886.943393:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+147:32834:x1876219350523392:12345-192.168.203.11@tcp:35:cat.0 00010000:00010000:2.0:1789302886.943404:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184f93e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0eef expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.943493:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.943543:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.943558:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.943579:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.943587:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.943596:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302886.944381:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+149:32834:x1876219350523392:12345-192.168.203.11@tcp:35:cat.0 Request processed in 993us (1226us total) trans 4294968295 rc 0/0 00000100:00100000:3.0:1789302886.944391:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 307 00010000:00010000:2.0:1789302886.944638:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.944651:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.944662:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302886.944674:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1789302886.946141:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946155:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946163:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946171:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.946175:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946184:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946200:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302886.946205:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302886.946210:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946216:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f953 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946241:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946247:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946253:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946258:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.946261:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946268:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946279:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.946282:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.946285:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946290:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f945 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946302:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946309:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946316:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946322:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.946326:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946333:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946340:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f10d3c600) 00010000:00010000:2.0:1789302886.946343:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.946345:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946351:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f94c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946412:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184f93e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eef expref: 149 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946423:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.946425:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302886.946429:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184f93e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0eef expref: 149 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.946439:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030000, rc 0) 00000100:00100000:2.0:1789302886.947609:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+148:32828:x1876219350523136:12345-192.168.203.11@tcp:101:cat.0 Request processed in 4273us (4530us total) trans 4294968296 rc 301/301 00000100:00100000:2.0:1789302886.947620:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1475 00000100:00100000:3.0:1789302886.952250:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.952335:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350523648 00000100:00100000:1.0:1789302886.952352:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350523648 00000100:00100000:1.0:1789302886.952436:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 308 00000100:00100000:1.0:1789302886.952470:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+148:32828:x1876219350523648:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302886.953931:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+149:32828:x1876219350523648:12345-192.168.203.11@tcp:35:cat.0 Request processed in 1452us (1677us total) trans 4294968297 rc 0/0 00000100:00100000:1.0:1789302886.953949:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 308 00000100:00100000:1.0:1789302886.961335:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302886.961393:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350523904 00000100:00100000:1.0:1789302886.961407:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350523904 00000100:00100000:1.0:1789302886.961821:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1476 00000100:00100000:1.0:1789302886.961841:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+149:32815:x1876219350523904:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302886.961872:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302886.961905:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f961 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ef6 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.962006:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.962024:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.962039:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.962057:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.962064:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.962072:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963157:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963176:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963189:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302886.963202:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302886.963315:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963323:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963331:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963338:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.963343:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963352:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963367:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302886.963371:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302886.963376:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963383:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184f976 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963405:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963412:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963419:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963424:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.963427:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963433:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963457:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.963461:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.963464:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963470:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184f968 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963480:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963486:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963492:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963498:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302886.963528:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963535:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963543:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302886.963545:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302886.963548:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963554:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184f96f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963613:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f961 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ef6 expref: 151 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963626:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302886.963629:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302886.963635:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184f961 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ef6 expref: 151 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302886.963647:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8c00, rc 0) 00000100:00100000:1.0:1789302886.963744:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+150:32815:x1876219350523904:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1911us (2425us total) trans 4294968298 rc 301/301 00000100:00100000:1.0:1789302886.963755:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1476 00000100:00100000:0.0:1789302886.971219:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.971285:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.971330:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350524416 00000100:00100000:0.0:1789302886.971344:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350524416 00000100:00100000:0.0:1789302886.971410:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1477 00000100:00100000:0.0:1789302886.971423:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+150:32831:x1876219350524416:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:0.0:1789302886.971474:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302886.971536:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f984 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0efd expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.971646:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.971666:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.971681:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.971700:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.971708:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.971716:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302886.971919:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350524160 00000100:00100000:3.0:1789302886.971926:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350524160 00000100:00100000:3.0:1789302886.971972:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 309 00000100:00100000:3.0:1789302886.971980:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+152:32815:x1876219350524160:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302886.972244:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+153:32815:x1876219350524160:12345-192.168.203.11@tcp:35:cat.0 Request processed in 264us (1040us total) trans 4294968299 rc 0/0 00000100:00100000:3.0:1789302886.972253:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 309 00010000:00010000:0.0:1789302886.977707:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.977735:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.977751:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302886.977767:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1789302886.978151:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978162:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978171:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978178:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.978186:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978196:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978212:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302886.978216:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302886.978222:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978230:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f999 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978254:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978263:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978269:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978276:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.978279:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978286:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978295:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.978298:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.978302:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978308:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f98b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978320:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978328:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978334:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978341:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302886.978344:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978352:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978360:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f09ae1f00) 00010000:00010000:0.0:1789302886.978362:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302886.978366:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978373:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184f992 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978443:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f984 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0efd expref: 153 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978473:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302886.978475:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302886.978479:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184f984 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0efd expref: 153 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.978490:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00000100:00100000:0.0:1789302886.978705:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+152:32831:x1876219350524416:12345-192.168.203.11@tcp:101:cat.0 Request processed in 7284us (7417us total) trans 4294968300 rc 301/301 00000100:00100000:0.0:1789302886.978717:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1477 00000100:00100000:0.0:1789302886.987032:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302886.987088:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302886.987162:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350524800 00000100:00100000:3.0:1789302886.987176:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350524800 00000100:00100000:3.0:1789302886.987246:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 310 00000100:00100000:3.0:1789302886.987258:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+152:32831:x1876219350524800:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302886.987584:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+153:32831:x1876219350524800:12345-192.168.203.11@tcp:35:cat.0 Request processed in 326us (564us total) trans 4294968301 rc 0/0 00000100:00100000:3.0:1789302886.987594:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 310 00000040:00080000:0.0:1789302886.989244:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302886.989254:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302886.989308:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 45 count 4 00000040:00080000:0.0:1789302886.989315:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302886.989471:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63df80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371176064:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302886.989654:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302886.991162:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350524928 00000100:00100000:2.0:1789302886.991177:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350524928 00000100:00100000:2.0:1789302886.991257:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1478 00000100:00100000:2.0:1789302886.991270:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+153:32826:x1876219350524928:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302886.991296:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302886.991321:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184f9a7 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f04 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.991422:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.991442:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.991469:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.991487:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.991495:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.991765:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992194:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992208:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992218:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302886.992230:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1789302886.992342:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992351:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992359:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992365:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.992371:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992383:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992398:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302886.992402:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302886.992408:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992415:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9bc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992433:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992440:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992445:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992464:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.992467:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992474:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992482:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.992485:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.992488:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992558:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9ae lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992572:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992578:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992584:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992589:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302886.992592:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992599:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992606:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f10d3c600) 00010000:00010000:2.0:1789302886.992608:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302886.992611:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992616:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184f9b5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992677:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184f9a7 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f04 expref: 155 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992694:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302886.992696:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302886.992700:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184f9a7 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f04 expref: 155 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302886.992710:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030000, rc 0) 00000100:00100000:2.0:1789302886.992788:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+154:32826:x1876219350524928:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1520us (5698us total) trans 4294968302 rc 301/301 00000100:00100000:2.0:1789302886.992797:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1478 00000001:00080000:0.0:1789302886.998610:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968286 is committed 00000001:00080000:0.0:1789302886.998651:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968255 is committed 00000020:00080000:0.0:1789302886.998660:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 34 batchid updated 00000001:00080000:0.0:1789302886.998670:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968292 is committed 00000004:00080000:0.0:1789302886.998711:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 18 committed 00000001:00080000:0.0:1789302886.998723:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968256 is committed 00000100:00100000:0.0:1789302886.998989:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371176192:0@lo:6:osp-syn-1-0.0 00000100:00100000:0.0:1789302886.999071:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302886.999146:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371176192 00000100:00100000:0.0:1789302886.999154:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371176192 00000100:00100000:0.0:1789302886.999213:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1789302886.999222:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371176192:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1789302886.999253:0:18578:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:29 count 1 00010000:00010000:0.0:1789302886.999285:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.999311:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.999322:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.999330:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789302886.999335:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302886.999341:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 5/0,1 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.999368:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.999384:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 17 pid: 6695 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302886.999396:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371176320:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302886.999485:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 11 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302886.999495:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 11 pid: 6696 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302886.999642:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371176448:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302886.999671:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efb6480 going to sleep for 11 seconds 00000100:00100000:0.0:1789302887.000186:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63df80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371176064:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302887.000400:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.000428:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350525056 00000100:00100000:0.0:1789302887.000433:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350525056 00000100:00100000:0.0:1789302887.000492:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1479 00000100:00100000:0.0:1789302887.000501:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32837:x1876219350525056:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000020:00080000:1.0:1789302887.000641:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 34 00000020:00080000:1.0:1789302887.000658:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 34 00000004:00000080:0.0:1789302887.000783:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x52:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302887.000993:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32837:x1876219350525056:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 491us (592us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302887.001002:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1479 00000100:00080000:2.0:1789302887.001287:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302887.001876:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.002018:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350525312 00000100:00100000:2.0:1789302887.002026:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350525312 00000100:00100000:2.0:1789302887.002086:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00100000:2.0:1789302887.002097:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076e8050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:32826:x1876219350525312:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:2.0:1789302887.002355:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076e8050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:32826:x1876219350525312:12345-192.168.203.11@tcp:35:cat.0 Request processed in 259us (481us total) trans 4294968303 rc 0/0 00000100:00100000:2.0:1789302887.002364:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 311 00000100:00100000:0.0:1789302887.003253:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.003668:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371176448:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302887.003999:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350525568 00000100:00100000:2.0:1789302887.004009:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350525568 00000100:00100000:2.0:1789302887.004424:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1480 00000100:00100000:2.0:1789302887.004438:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:32824:x1876219350525568:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302887.004474:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302887.004532:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9d1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f0b expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.004634:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.004652:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.004666:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.004683:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.004691:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.004699:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005229:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005242:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005250:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302887.005261:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1789302887.005369:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005383:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005390:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005395:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302887.005400:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005408:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005423:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302887.005427:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302887.005433:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005439:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184f9e6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005468:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005475:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005480:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005486:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302887.005490:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005495:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005527:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302887.005529:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302887.005532:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005538:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184f9d8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005547:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005554:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005559:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005564:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302887.005567:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005573:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005580:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f124a9300) 00010000:00010000:2.0:1789302887.005583:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302887.005586:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005591:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184f9df lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005645:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9d1 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f0b expref: 120 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005658:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:2.0:1789302887.005660:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302887.005664:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184f9d1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f0b expref: 120 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.005674:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031800, rc 0) 00000100:00100000:2.0:1789302887.005766:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+119:32824:x1876219350525568:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1331us (2516us total) trans 4294968304 rc 301/301 00000100:00100000:2.0:1789302887.005776:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1480 00000100:00100000:3.0:1789302887.006960:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.007003:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.007295:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.007391:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371176320:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302887.007431:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.010062:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350525696 00000100:00100000:3.0:1789302887.010083:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350525696 00010000:00010000:3.0:1789302887.010161:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 12 pid: 6696 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.010290:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350525824 00000100:00100000:3.0:1789302887.010294:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350525824 00010000:00010000:3.0:1789302887.010310:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 18 pid: 6695 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.010335:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00100000:3.0:1789302887.010346:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350525696:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302887.010364:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302887.010370:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 12 pid: 6696 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010384:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 12 pid: 6696 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010415:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000da7bf650/0x64fe43095184f651 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e32 expref: 12 pid: 6696 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010442:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302887.010558:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350525696:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 213us (3670us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.010565:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00100000:3.0:1789302887.010584:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:3.0:1789302887.010591:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350525824:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302887.010600:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302887.010605:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 18 pid: 6695 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010616:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 18 pid: 6695 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010631:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000af4abfe8/0x64fe43095184f674 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1d:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e40 expref: 18 pid: 6695 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010642:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302887.010646:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1d:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302887.010653:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010663:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.010698:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302887.010719:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350525824:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 128us (3715us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.010726:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:3.0:1789302887.010814:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.011027:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.011355:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350525952 00000100:00100000:0.0:1789302887.011366:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350525952 00000100:00100000:0.0:1789302887.011420:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350526208 00000100:00100000:0.0:1789302887.011424:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350526208 00000100:00100000:0.0:1789302887.011445:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1481 00000100:00100000:0.0:1789302887.011608:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32837:x1876219350525952:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 00010000:00010000:0.0:1789302887.011698:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 3/1,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011717:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011730:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011748:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 3/1,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011756:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011762:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011791:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011799:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.011806:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302887.011856:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00000004:00200000:0.0:1789302887.011860:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302887.011876:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.012281:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1482 00000100:00100000:3.0:1789302887.012295:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+119:32822:x1876219350526208:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302887.012316:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.012330:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fa09 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f12 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.012394:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.012404:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.012414:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.012428:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.012436:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.012442:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012603:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 3/1,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012613:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 2/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012621:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012627:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.012632:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 3/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012642:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 4/0,0 mode: CR/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012655:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:0.0:1789302887.012659:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302887.012665:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012672:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184f9f4 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012690:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 3/1,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012697:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 2/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012713:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012719:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.012722:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 3/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012728:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 4/0,0 mode: PR/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012736:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x8138294e (ffff960f09ae1000) 00010000:00010000:0.0:1789302887.012739:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302887.012742:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 2/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012747:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184f9fb lrc: 0/0,0 mode: --/PR res: [0x200000002:0x2:0x0].0x8138294e bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012765:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012772:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012778:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012783:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.012786:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012792:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012803:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302887.012805:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302887.012808:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.012814:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fa02 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789302887.013072:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1789302887.013423:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013438:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013448:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.013470:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302887.013606:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013614:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013620:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013626:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.013629:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013636:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013645:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.013647:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302887.013651:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013662:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fa1e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013674:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013680:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013686:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013690:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.013693:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013699:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013706:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.013708:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.013711:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013717:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fa10 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013725:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013732:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013739:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013744:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.013747:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013752:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013759:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302887.013761:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.013763:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013769:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fa17 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013816:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fa09 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f12 expref: 121 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013828:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.013831:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302887.013834:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fa09 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f12 expref: 121 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.013844:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:0.0:1789302887.013995:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32837:x1876219350525952:12345-192.168.203.11@tcp:34:lhsmtool_posix.0 Request processed in 2390us (6698us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302887.014008:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1481 00000100:00100000:0.0:1789302887.014067:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350526464 00000100:00100000:0.0:1789302887.014075:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350526464 00000100:00100000:0.0:1789302887.014110:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 312 00000100:00100000:0.0:1789302887.014118:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+121:32824:x1876219350526464:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302887.014323:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32822:x1876219350526208:12345-192.168.203.11@tcp:101:cat.0 Request processed in 2033us (3509us total) trans 4294968305 rc 301/301 00000100:00100000:3.0:1789302887.014334:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1482 00010000:00010000:3.0:1789302887.015127:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.015143:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.015155:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.015162:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.015186:0:18578:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302887.016135:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371176192:12345-0@lo:6:osp-syn-1-0.0 Request processed in 16904us (17054us total) trans 4294967381 rc 0/0 00000100:00100000:3.0:1789302887.016147:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:0.0:1789302887.017490:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+121:32824:x1876219350526464:12345-192.168.203.11@tcp:35:cat.0 Request processed in 3358us (6583us total) trans 4294968306 rc 0/0 00000100:00100000:0.0:1789302887.017532:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 312 00000004:00080000:0.0:1789302887.017620:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f63e300/2, rc 0, transno 4294967381 00000100:00100000:0.0:1789302887.017629:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371176192:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1789302887.018643:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302887.018658:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.018675:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302887.018680:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.018693:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.018721:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 1/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.018730:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000066c20e73/0x64fe43095184f9ca lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.018761:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302887.018763:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302887.018765:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302887.022769:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.022810:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.023256:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350526976 00000100:00100000:2.0:1789302887.023271:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350526976 00000100:00100000:2.0:1789302887.023342:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 313 00000100:00100000:2.0:1789302887.023354:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+121:32822:x1876219350526976:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:2.0:1789302887.023696:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32822:x1876219350526976:12345-192.168.203.11@tcp:35:cat.0 Request processed in 342us (882us total) trans 4294968307 rc 0/0 00000100:00100000:2.0:1789302887.023704:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 313 00000100:00100000:2.0:1789302887.023863:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350526720 00000100:00100000:2.0:1789302887.023870:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350526720 00000100:00100000:2.0:1789302887.024275:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1483 00000100:00100000:2.0:1789302887.024285:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32837:x1876219350526720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302887.024302:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302887.024327:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fa2c lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f19 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024425:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024444:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024471:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302887.024484:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:2.0:1789302887.024607:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024615:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024624:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024630:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302887.024635:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024643:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024661:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302887.024664:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302887.024671:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024677:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184fa33 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024722:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fa2c lrc: 1/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f19 expref: 52 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024733:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x2:0x0].0x0 (ffff960f11d36400) 00010000:00010000:2.0:1789302887.024735:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1789302887.024739:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fa2c lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f19 expref: 52 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.024751:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031c00, rc 0) 00000100:00100000:2.0:1789302887.024809:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32837:x1876219350526720:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 526us (2056us total) trans 4294968308 rc 301/301 00000100:00100000:2.0:1789302887.024817:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1483 00000100:00100000:2.0:1789302887.026851:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.028164:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350527232 00000100:00100000:3.0:1789302887.028182:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350527232 00000100:00100000:3.0:1789302887.028266:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1484 00000100:00100000:3.0:1789302887.028279:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+122:32832:x1876219350527232:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302887.028306:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.028335:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fa41 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f20 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.028437:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.028469:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.028483:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.028532:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.028540:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.028546:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029648:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029666:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029677:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.029692:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302887.029817:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029827:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029836:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029843:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.029849:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029858:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029872:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.029875:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302887.029881:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.029888:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fa56 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030174:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030181:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030187:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030193:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.030196:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030203:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030217:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.030220:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.030224:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030230:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fa48 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030239:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030245:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030251:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030357:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.030362:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030369:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030378:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2800) 00010000:00010000:3.0:1789302887.030381:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.030384:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.030390:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fa4f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.031994:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.032082:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350527360 00000100:00100000:2.0:1789302887.032095:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350527360 00000100:00100000:2.0:1789302887.032179:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1485 00000100:00100000:2.0:1789302887.032191:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32837:x1876219350527360:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302887.032241:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000401:0x52:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302887.032388:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32837:x1876219350527360:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 198us (407us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302887.032398:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1485 00010000:00010000:1.0:1789302887.034245:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fa41 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f20 expref: 124 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.034282:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302887.034286:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302887.034292:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fa41 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f20 expref: 124 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.034311:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:1.0:1789302887.034392:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+123:32832:x1876219350527232:12345-192.168.203.11@tcp:101:cat.0 Request processed in 6115us (7561us total) trans 4294968309 rc 301/301 00000100:00100000:1.0:1789302887.034401:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1484 00000100:00100000:3.0:1789302887.036234:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.036880:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350527616 00000100:00100000:0.0:1789302887.036961:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350527616 00000100:00100000:0.0:1789302887.037039:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1486 00000100:00100000:0.0:1789302887.037052:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:32837:x1876219350527616:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302887.037082:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302887.037205:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184fa64 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f27 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302887.038395:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1e:0x0], index 0: 0 00010000:00010000:0.0:1789302887.041022:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fa6b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.041046:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fa6b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.041059:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fa6b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.041175:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184fa64 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f27 expref: 54 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.041332:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fa6b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f27 expref: 53 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.041341:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f09ae1000) 00010000:00010000:0.0:1789302887.041345:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302887.041349:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc 0) 00000100:00100000:0.0:1789302887.041430:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32837:x1876219350527616:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 4381us (5207us total) trans 4294968310 rc 0/0 00000100:00100000:0.0:1789302887.041439:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1486 00000100:00100000:0.0:1789302887.045420:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.045568:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.045911:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350528128 00000100:00100000:3.0:1789302887.045925:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350528128 00000100:00100000:3.0:1789302887.045988:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1487 00000100:00100000:3.0:1789302887.046000:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+123:32827:x1876219350528128:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302887.046033:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.046060:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fa79 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f2e expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046171:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046189:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046203:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046221:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046229:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046237:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046704:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046716:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046735:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.046748:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302887.046858:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046867:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046874:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046881:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.046887:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046896:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046909:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.046912:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302887.046918:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046924:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fa8e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046948:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046955:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046961:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046967:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.046970:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046978:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046985:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.046988:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.046991:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.046997:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fa80 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047005:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047011:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047017:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047023:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.047026:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047032:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047039:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302887.047041:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.047044:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047050:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fa87 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047102:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fa79 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f2e expref: 125 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047113:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.047115:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302887.047119:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fa79 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f2e expref: 125 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.047128:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:3.0:1789302887.047212:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+124:32827:x1876219350528128:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1213us (1652us total) trans 4294968311 rc 301/301 00000100:00100000:3.0:1789302887.047221:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1487 00000100:00100000:1.0:1789302887.047794:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350527872 00000100:00100000:1.0:1789302887.047806:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350527872 00000100:00100000:1.0:1789302887.047860:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 314 00000100:00100000:1.0:1789302887.047870:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+124:32832:x1876219350527872:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:1.0:1789302887.048662:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+125:32832:x1876219350527872:12345-192.168.203.11@tcp:35:cat.0 Request processed in 789us (3248us total) trans 4294968312 rc 0/0 00000100:00100000:1.0:1789302887.048672:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 314 00010000:00010000:2.0:1789302887.054440:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302887.058082:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.058377:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350528384 00000100:00100000:1.0:1789302887.058391:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350528384 00000100:00100000:1.0:1789302887.058668:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1488 00000100:00100000:1.0:1789302887.058682:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32837:x1876219350528384:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:1.0:1789302887.058774:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302887.058811:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fa6b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f27 expref: 53 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.058840:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f09ae1000) 00010000:00010000:1.0:1789302887.058854:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302887.058884:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 3/0,1 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.058913:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.058924:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.058942:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.058949:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.058955:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059349:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059359:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059366:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059372:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302887.059377:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059387:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059398:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302887.059401:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302887.059406:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059412:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095184fa9c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059423:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059428:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059434:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059439:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302887.059442:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059459:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059467:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x7c447202 (ffff960f2c94d800) 00010000:00010000:1.0:1789302887.059470:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302887.059473:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.059483:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184faa3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x51:0x0].0x7c447202 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.060189:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32837:x1876219350528384:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1503us (2114us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302887.060201:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1488 00000100:00100000:0.0:1789302887.065334:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.065402:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.065433:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350528640 00000100:00100000:2.0:1789302887.065461:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350528640 00000100:00100000:3.0:1789302887.065553:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350528896 00000100:00100000:3.0:1789302887.065564:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350528896 00000100:00100000:2.0:1789302887.065588:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 315 00000100:00100000:2.0:1789302887.065605:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+126:32827:x1876219350528640:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302887.065632:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1489 00000100:00100000:3.0:1789302887.065645:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+126:32817:x1876219350528896:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302887.065668:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.065690:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184faaa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f35 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.065780:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.065794:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.065807:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.065824:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.065831:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.065840:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.065967:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+128:32827:x1876219350528640:12345-192.168.203.11@tcp:35:cat.0 Request processed in 362us (639us total) trans 4294968313 rc 0/0 00000100:00100000:2.0:1789302887.065978:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 315 00010000:00010000:3.0:1789302887.066249:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066261:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066269:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.066280:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302887.066369:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066376:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066383:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066390:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.066395:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066402:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066416:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.066420:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302887.066424:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066431:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fabf lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.066443:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302887.066459:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066470:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066481:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066486:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00000100:00100000:2.0:1789302887.066488:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350529024 00010000:00010000:3.0:1789302887.066490:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066597:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066607:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.066610:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.066614:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066619:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fab1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066629:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.066631:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350529024 00010000:00010000:3.0:1789302887.066640:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066648:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066654:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.066657:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066668:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066676:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302887.066681:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.066685:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066694:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fab8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066738:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184faaa lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f35 expref: 128 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.066749:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.066752:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302887.066756:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184faaa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f35 expref: 128 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.066760:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1490 00010000:00010000:3.0:1789302887.066773:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:2.0:1789302887.066781:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32837:x1876219350529024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302887.066799:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302887.066825:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184facd lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f3c expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.066842:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+127:32817:x1876219350528896:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1197us (1436us total) trans 4294968314 rc 301/301 00000100:00100000:3.0:1789302887.066852:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1489 00010000:00010000:2.0:1789302887.066862:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.066878:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.067201:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.067324:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030400/0x64fe43095184facd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 54 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.067344:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 53 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.067353:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:2.0:1789302887.067356:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302887.067359:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030800, rc 0) 00000100:00100000:2.0:1789302887.067444:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb9500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32837:x1876219350529024:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 670us (1003us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302887.067460:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1490 00000100:00100000:1.0:1789302887.071457:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.071840:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350529280 00000100:00100000:0.0:1789302887.071856:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350529280 00000100:00100000:0.0:1789302887.072003:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1491 00000100:00100000:0.0:1789302887.072016:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32837:x1876219350529280:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:0.0:1789302887.072197:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.072219:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.072235:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.072687:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.073597:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350529792 00000100:00100000:1.0:1789302887.073610:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350529792 00000100:00100000:3.0:1789302887.073642:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.073666:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 316 00000100:00100000:3.0:1789302887.073678:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350529536 00000100:00100000:1.0:1789302887.073679:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+127:32817:x1876219350529792:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:3.0:1789302887.073684:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350529536 00000100:00100000:3.0:1789302887.073726:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1492 00000100:00100000:3.0:1789302887.073734:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+128:32833:x1876219350529536:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302887.073787:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.073812:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fae2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f43 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.073892:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.073908:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.073920:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.073937:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.073944:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.073950:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.074216:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+130:32817:x1876219350529792:12345-192.168.203.11@tcp:35:cat.0 Request processed in 539us (1529us total) trans 4294968315 rc 0/0 00000100:00100000:1.0:1789302887.074226:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 316 00000040:00080000:0.0:1789302887.074474:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].18, 88 off9416 00000004:00080000:0.0:1789302887.074497:0:5844:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 724084993666ns, next at 725084989289ns, rc = 0 00010000:00010000:3.0:1789302887.074531:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074544:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074554:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.074567:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302887.074751:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074760:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074767:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074773:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.074778:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074788:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074803:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.074806:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302887.074812:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074819:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184faf7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074836:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302887.074840:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ded43072 reply state 00000000072e9992 transno 4294968316 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074843:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074861:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.074862:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302887.074875:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.074880:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074886:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074894:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.074897:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.074899:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074905:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fae9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074914:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074921:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074927:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074932:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.074934:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074940:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074950:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302887.074952:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.074955:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.074962:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184faf0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.075007:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fae2 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f43 expref: 130 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.075019:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.075021:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302887.075026:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fae2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f43 expref: 130 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.075037:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:0.0:1789302887.075132:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32837:x1876219350529280:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 3118us (3697us total) trans 4294968316 rc 0/0 00000100:00100000:0.0:1789302887.075140:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1491 00000100:00100000:3.0:1789302887.075537:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:32833:x1876219350529536:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1802us (1898us total) trans 4294968317 rc 301/301 00000100:00100000:3.0:1789302887.075544:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1492 00010000:00010000:1.0:1789302887.081329:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.081351:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.081361:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.081367:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302887.081373:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.081382:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.081405:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:1.0:1789302887.081408:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1789302887.081414:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.081421:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fadb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.084289:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.085780:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.087738:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350530048 00000100:00100000:3.0:1789302887.087759:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350530048 00000100:00100000:3.0:1789302887.087823:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350530176 00000100:00100000:3.0:1789302887.087827:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350530176 00000100:00100000:3.0:1789302887.087876:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1493 00000100:00100000:3.0:1789302887.087888:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+129:32830:x1876219350530048:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302887.087924:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:0.0:1789302887.088055:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1494 00000100:00100000:0.0:1789302887.088067:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32837:x1876219350530176:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302887.088082:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302887.088101:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f51 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.088180:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.088202:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.088225:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:0.0:1789302887.088229:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302887.088234:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a800, rc 0) 00000100:00100000:0.0:1789302887.088305:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32837:x1876219350530176:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 241us (2536us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302887.088313:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1494 00000100:00100000:2.0:1789302887.089326:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.089545:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350530432 00000100:00100000:2.0:1789302887.089555:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350530432 00000100:00100000:2.0:1789302887.089612:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 317 00000100:00100000:2.0:1789302887.089622:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076eac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+130:32833:x1876219350530432:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:2.0:1789302887.090326:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076eac50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+131:32833:x1876219350530432:12345-192.168.203.11@tcp:35:cat.0 Request processed in 700us (1004us total) trans 4294968318 rc 0/0 00000100:00100000:2.0:1789302887.090337:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 317 00010000:00010000:3.0:1789302887.091021:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fb0c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f4a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.091134:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.091153:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.091166:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.091184:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.091191:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.091197:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.098314:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.098345:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.098361:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.098377:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302887.099178:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099192:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099200:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099207:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.099213:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099221:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099238:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.099242:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302887.099248:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099255:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095184fb21 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099272:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099278:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099283:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099291:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.099293:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099298:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099305:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.099308:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.099310:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099315:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095184fb13 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099324:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099329:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099335:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099340:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.099342:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099348:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099361:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2400) 00010000:00010000:3.0:1789302887.099364:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.099366:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099373:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095184fb1a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099457:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fb0c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f4a expref: 132 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099470:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.099748:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302887.099756:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095184fb0c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f4a expref: 132 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.099768:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:3.0:1789302887.100202:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+131:32830:x1876219350530048:12345-192.168.203.11@tcp:101:cat.0 Request processed in 12315us (15929us total) trans 4294968319 rc 301/301 00000100:00100000:3.0:1789302887.100215:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1493 00000100:00100000:3.0:1789302887.105374:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.105424:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.105480:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.105649:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350530560 00000100:00100000:1.0:1789302887.105663:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350530560 00000100:00100000:1.0:1789302887.105759:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 415 00000100:00100000:1.0:1789302887.105773:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350530560:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00000100:00100000:0.0:1789302887.106056:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350531072 00000100:00100000:0.0:1789302887.106063:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350531072 00000100:00100000:0.0:1789302887.106208:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1495 00000100:00100000:0.0:1789302887.106219:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+131:32836:x1876219350531072:12345-192.168.203.11@tcp:101:cat.0 00000100:00100000:1.0:1789302887.106233:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350530560:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 463us (807us total) trans 4294967380 rc 0/0 00010000:00010000:0.0:1789302887.106242:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1789302887.106243:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 415 00010000:00010000:0.0:1789302887.106266:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184fb2f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f58 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.106276:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350530816 00000100:00100000:1.0:1789302887.106281:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350530816 00000100:00100000:1.0:1789302887.106310:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 318 00000100:00100000:1.0:1789302887.106318:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+133:32830:x1876219350530816:12345-192.168.203.11@tcp:35:cat.0 00010000:00010000:0.0:1789302887.106356:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106376:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106391:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106410:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106419:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106428:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.106478:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+134:32830:x1876219350530816:12345-192.168.203.11@tcp:35:cat.0 Request processed in 160us (996us total) trans 4294968320 rc 0/0 00000100:00100000:1.0:1789302887.106486:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 318 00010000:00010000:0.0:1789302887.106850:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106861:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106869:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302887.106881:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:0.0:1789302887.106942:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106950:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106958:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106965:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.106970:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.106979:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302887.107622:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:0.0:1789302887.108176:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302887.108182:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302887.108191:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108201:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fb44 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108227:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108234:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108241:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108247:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.108250:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108258:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108266:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302887.108268:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302887.108271:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108277:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fb36 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108292:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108299:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108304:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108309:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.108312:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108317:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108324:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f09ae1f00) 00010000:00010000:0.0:1789302887.108327:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302887.108329:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108334:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fb3d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108404:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184fb2f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f58 expref: 134 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108416:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:0.0:1789302887.108419:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1789302887.108424:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184fb2f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f58 expref: 134 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.108441:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc 0) 00000100:00100000:0.0:1789302887.108536:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+133:32836:x1876219350531072:12345-192.168.203.11@tcp:101:cat.0 Request processed in 2319us (3175us total) trans 4294968321 rc 301/301 00000100:00100000:0.0:1789302887.108545:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1495 00000100:00100000:0.0:1789302887.113720:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.113989:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350531328 00000100:00100000:3.0:1789302887.114003:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350531328 00000100:00100000:3.0:1789302887.114089:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1496 00000100:00100000:3.0:1789302887.114116:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+133:32825:x1876219350531328:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:3.0:1789302887.114155:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.114187:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fb52 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f5f expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114307:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114327:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114342:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114361:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114368:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114375:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114854:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114868:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.114876:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.114887:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:3.0:1789302887.115000:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115009:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115017:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115023:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.115028:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115037:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115051:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.115055:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302887.115061:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115067:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fb67 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115085:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115091:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115098:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115108:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.115111:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115117:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115125:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.115128:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.115131:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115137:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fb59 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115146:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115152:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115159:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115164:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302887.115167:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115173:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115185:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f0cdf2800) 00010000:00010000:3.0:1789302887.115188:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302887.115190:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115195:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb60 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115249:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fb52 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f5f expref: 135 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115260:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:3.0:1789302887.115262:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:3.0:1789302887.115266:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fb52 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f5f expref: 135 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.115275:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302887.115366:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+134:32825:x1876219350531328:12345-192.168.203.11@tcp:101:cat.0 Request processed in 1265us (1664us total) trans 4294968322 rc 301/301 00000100:00100000:3.0:1789302887.115374:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1496 00000100:00100000:0.0:1789302887.124910:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.124970:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.125054:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.125066:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350532096 00000100:00100000:1.0:1789302887.125080:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350532096 00000100:00100000:0.0:1789302887.125081:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350531584 00000100:00100000:0.0:1789302887.125087:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350531584 00000100:00100000:1.0:1789302887.125146:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1497 00000100:00100000:1.0:1789302887.125159:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+135:32835:x1876219350532096:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:1.0:1789302887.125191:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302887.125217:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184fb75 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f66 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.125237:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350531840 00000100:00100000:0.0:1789302887.125242:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350531840 00000100:00100000:0.0:1789302887.125262:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 319 00000100:00100000:0.0:1789302887.125271:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+136:32836:x1876219350531584:12345-192.168.203.11@tcp:35:cat.0 00010000:00010000:1.0:1789302887.125392:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.125412:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.125427:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.125554:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.125563:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.125570:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.125591:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+137:32836:x1876219350531584:12345-192.168.203.11@tcp:35:cat.0 Request processed in 320us (695us total) trans 4294968323 rc 0/0 00000100:00100000:2.0:1789302887.125734:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:2.0:1789302887.125750:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:32837:x1876219350531840:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302887.125865:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32837:x1876219350531840:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 121us (810us total) trans 4294968324 rc 0/0 00000100:00100000:2.0:1789302887.125874:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 320 00000100:00100000:0.0:1789302887.127281:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 319 00010000:00010000:1.0:1789302887.127889:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.127911:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.127938:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302887.127954:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x52:0x0]: returning size 158 00010000:00010000:1.0:1789302887.128078:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128088:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128096:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128104:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302887.128110:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128119:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128133:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302887.128137:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302887.128142:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128149:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095184fb8a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128170:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128177:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128185:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128195:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302887.128199:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128205:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128211:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302887.128214:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302887.128216:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128222:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe43095184fb7c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128231:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128237:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128242:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128247:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302887.128250:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128256:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128267:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x2e0f5ada (ffff960f2c94d800) 00010000:00010000:1.0:1789302887.128270:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302887.128272:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128283:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095184fb83 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x51:0x0].0x2e0f5ada bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128341:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184fb75 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f66 expref: 137 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128353:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x51:0x0].0x0 (ffff960f09ae1100) 00010000:00010000:1.0:1789302887.128355:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:1.0:1789302887.128359:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095184fb75 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x51:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f66 expref: 137 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.128368:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8a00, rc 0) 00000100:00100000:1.0:1789302887.128484:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+136:32835:x1876219350532096:12345-192.168.203.11@tcp:101:cat.0 Request processed in 3327us (3513us total) trans 4294968325 rc 301/301 00000100:00100000:1.0:1789302887.128493:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1497 00000100:00100000:0.0:1789302887.129800:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.130476:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350532352 00000100:00100000:1.0:1789302887.130489:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350532352 00000100:00100000:1.0:1789302887.130583:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1498 00000100:00100000:1.0:1789302887.130595:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32837:x1876219350532352:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302887.130727:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32837:x1876219350532352:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 133us (934us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302887.130739:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1498 00000100:00100000:2.0:1789302887.150042:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.150289:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350532608 00000100:00100000:0.0:1789302887.150304:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350532608 00000100:00100000:0.0:1789302887.150377:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:0.0:1789302887.150388:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+136:32825:x1876219350532608:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:0.0:1789302887.150975:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+137:32825:x1876219350532608:12345-192.168.203.11@tcp:35:cat.0 Request processed in 586us (957us total) trans 4294968326 rc 0/0 00000100:00100000:0.0:1789302887.150989:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 321 00000100:00100000:0.0:1789302887.153884:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.155381:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350532736 00000100:00100000:3.0:1789302887.155399:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350532736 00000100:00100000:3.0:1789302887.155493:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1499 00000100:00100000:3.0:1789302887.155546:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32837:x1876219350532736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302887.155575:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.155721:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fb98 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f6d expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.155831:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb9f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.155851:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb9f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.155866:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb9f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302887.155916:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302887.155941:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb9f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.155994:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fb98 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f6d expref: 58 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.156018:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb9f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f6d expref: 57 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.156275:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:3.0:1789302887.156278:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1a 00010000:00010000:3.0:1789302887.156283:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302887.156367:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32837:x1876219350532736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 863us (2496us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.156376:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1499 00000100:00100000:2.0:1789302887.156621:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.156906:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350532992 00000100:00100000:0.0:1789302887.156915:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350532992 00000100:00100000:0.0:1789302887.156966:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 322 00000100:00100000:0.0:1789302887.156984:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+137:32835:x1876219350532992:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:0.0:1789302887.157253:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+138:32835:x1876219350532992:12345-192.168.203.11@tcp:35:cat.0 Request processed in 270us (631us total) trans 4294968327 rc 0/0 00000100:00100000:0.0:1789302887.157261:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 322 00000100:00100000:2.0:1789302887.160553:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.160716:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350533120 00000100:00100000:3.0:1789302887.160729:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350533120 00000100:00100000:3.0:1789302887.160795:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 416 00000100:00100000:3.0:1789302887.160807:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350533120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302887.161283:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302887.161309:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f74 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302887.161342:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x6aa69866, blocks: 0x1 00010000:00010000:3.0:1789302887.161365:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=724s v=(0 0) 00010000:00010000:3.0:1789302887.161370:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302887.161381:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.161401:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.161416:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302887.161499:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350533120:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 693us (958us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.161540:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 416 00000100:00100000:1.0:1789302887.177309:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.177415:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350533376 00000100:00100000:3.0:1789302887.177430:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350533376 00000100:00100000:3.0:1789302887.177558:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1500 00000100:00100000:3.0:1789302887.177572:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32837:x1876219350533376:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302887.177728:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32837:x1876219350533376:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 159us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.177738:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1500 00000100:00100000:1.0:1789302887.186927:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.187392:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350533504 00000100:00100000:2.0:1789302887.187419:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350533504 00000100:00100000:2.0:1789302887.188226:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 417 00000100:00100000:2.0:1789302887.188252:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:32837:x1876219350533504:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302887.188300:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302887.188337:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f7b expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.188405:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.188436:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302887.188457:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302887.188465:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.188687:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.188706:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 12 pid: 6697 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.188841:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371176576:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302887.189336:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fc66540 going to sleep for 11 seconds 00000100:00100000:2.0:1789302887.198634:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.199734:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350533632 00000100:00100000:3.0:1789302887.199768:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350533632 00010000:00010000:3.0:1789302887.199862:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 13 pid: 6697 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.200040:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:3.0:1789302887.200068:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350533632:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302887.200127:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302887.200171:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 13 pid: 6697 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.200201:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 13 pid: 6697 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.200258:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302887.200264:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302887.200287:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.200318:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.200416:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.200479:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371176704:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302887.201213:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75000 going to sleep for 11 seconds 00010000:00010000:0.0:1789302887.203899:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d356e686/0x64fe43095184fba6 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f74 expref: 13 pid: 6697 timeout: 824 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.203999:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb480 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371176576:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302887.204041:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 12 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.204060:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031400, rc 0) 00000100:00100000:0.0:1789302887.204270:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:32837:x1876219350533504:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 16019us (17367us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302887.204281:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 417 00000100:00100000:0.0:1789302887.208320:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31f80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371176704:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302887.208357:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302887.208428:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350533632:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 8363us (9821us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302887.208459:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:1.0:1789302887.220266:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.221006:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350533760 00000100:00100000:3.0:1789302887.221035:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350533760 00002000:00100000:3.0:1789302887.221156:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1e:0x0] (0->157) req@ffff960f095adc00 x1876219350533760/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:463/0 lens 648/0 e 0 to 0 dl 1789302898 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302887.221184:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f095adc00 with x1876219350533760 ext(0->157) 00002000:00010000:3.0:1789302887.221191:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f095adc00 00000100:00100000:3.0:1789302887.221307:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 126 00000100:00100000:3.0:1789302887.221334:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350533760:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 00000100:00100000:3.0:1789302887.222596:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095adc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350533760:12345-192.168.203.11@tcp:4:lhsmtool_posix.0 Request processed in 1255us (2350us total) trans 4294967381 rc 0/0 00000100:00100000:3.0:1789302887.222624:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 126 00002000:00100000:3.0:1789302887.222636:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x1e:0x0] (0->157) req@ffff960f095adc00 x1876219350533760/t4294967381(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:463/0 lens 648/448 e 0 to 0 dl 1789302898 ref 1 fl Complete:/600/0 rc 0/0 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302887.222661:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f095adc00 with x1876219350533760 ext(0->157) 00002000:00010000:3.0:1789302887.222667:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f095adc00 00000100:00100000:1.0:1789302887.231720:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.231966:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350533888 00000100:00100000:3.0:1789302887.232016:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350533888 00000100:00100000:3.0:1789302887.232171:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1501 00000100:00100000:3.0:1789302887.232199:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:32837:x1876219350533888:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 00000001:00080000:3.0:1789302887.334317:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968327 is committed 00000001:00080000:3.0:1789302887.334359:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968316 is committed 00000004:00080000:3.0:1789302887.334381:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 20 committed 00000100:00100000:0.0:1789302887.335998:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371176832:0@lo:2:osp-syn-0-0.0 00000100:00100000:0.0:1789302887.336180:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302887.336438:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371176832 00000100:00100000:3.0:1789302887.336471:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371176832 00000100:00100000:3.0:1789302887.336648:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:3.0:1789302887.336659:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371176832:12345-0@lo:2:osp-syn-0-0.0 00002000:00010000:3.0:1789302887.337079:0:8571:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb size from disk: 0 -> 158 00002000:00010000:3.0:1789302887.337085:0:8571:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb mtime from disk: 0 -> 1789302886 00002000:00010000:3.0:1789302887.337099:0:8571:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb atime from disk: 0 -> 0 00002000:00010000:3.0:1789302887.337102:0:8571:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb ctime from disk: 1789302886 -> 1789302886 00002000:00010000:3.0:1789302887.337104:0:8571:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb blocks from disk: 1 -> 1 00000100:00100000:3.0:1789302887.337202:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371176832:12345-0@lo:2:osp-syn-0-0.0 Request processed in 532us (1024us total) trans 4294967382 rc 0/0 00000100:00100000:3.0:1789302887.337211:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000004:00080000:3.0:1789302887.337400:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3fe30a80/2, rc 0, transno 4294967382 00000100:00100000:3.0:1789302887.337408:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe30a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371176832:0@lo:2:osp-syn-0-0.0 00000100:00080000:0.0:1789302887.337561:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789302887.338080:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968324 is committed 00000100:00100000:0.0:1789302887.338677:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32837:x1876219350533888:12345-192.168.203.11@tcp:44:lhsmtool_posix.0 Request processed in 106476us (106989us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302887.338702:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1501 00000100:00100000:1.0:1789302887.343174:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.343364:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350534016 00000100:00100000:2.0:1789302887.343378:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350534016 00000100:00100000:2.0:1789302887.343544:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 419 00000100:00100000:2.0:1789302887.343574:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350534016:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 00000001:00080000:0.0:1789302887.419258:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967380 is committed 00000001:00080000:0.0:1789302887.419291:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967382 is committed 00000001:00080000:0.0:1789302887.419319:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967381 is committed 00000100:00100000:2.0:1789302887.419576:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350534016:12345-192.168.203.11@tcp:16:lhsmtool_posix.0 Request processed in 75999us (76436us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302887.419590:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 419 00000100:00100000:1.0:1789302887.426908:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.430815:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350534272 00000100:00100000:3.0:1789302887.430866:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350534272 00000100:00100000:3.0:1789302887.431151:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1502 00000100:00100000:3.0:1789302887.431166:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32837:x1876219350534272:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 00010000:00010000:3.0:1789302887.431289:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302887.431356:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fb9f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x1a/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f6d expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.431395:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:3.0:1789302887.431402:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302887.431440:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.431467:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.431479:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302887.432126:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000002c017953 reply state 00000000e9cab663 transno 4294968328 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.432153:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302887.432738:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:32837:x1876219350534272:12345-192.168.203.11@tcp:36:lhsmtool_posix.0 Request processed in 1570us (5847us total) trans 4294968328 rc 0/0 00000100:00100000:3.0:1789302887.432748:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1502 00010000:00010000:2.0:1789302887.436432:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.436785:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.436795:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.436818:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302887.436824:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.436833:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.436883:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:2.0:1789302887.436889:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302887.436896:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.436916:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fbb4 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.439705:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.440210:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350534400 00000100:00100000:3.0:1789302887.440226:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350534400 00000100:00100000:3.0:1789302887.440344:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 420 00000100:00100000:3.0:1789302887.440357:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350534400:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 00002000:00010000:3.0:1789302887.440736:0:18578:0:(ofd_lvb.c:306:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb size from disk: 158 -> 158 00002000:00010000:3.0:1789302887.440741:0:18578:0:(ofd_lvb.c:313:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb mtime from disk: 1789302886 -> 1789302884 00002000:00010000:3.0:1789302887.440744:0:18578:0:(ofd_lvb.c:319:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb atime from disk: 0 -> 1789302884 00002000:00010000:3.0:1789302887.440747:0:18578:0:(ofd_lvb.c:325:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb ctime from disk: 1789302886 -> 1789302886 00002000:00010000:3.0:1789302887.440750:0:18578:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x240000400:0x1e:0x0] updating lvb blocks from disk: 1 -> 11 00000100:00100000:3.0:1789302887.441120:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350534400:12345-192.168.203.11@tcp:2:lhsmtool_posix.0 Request processed in 763us (1623us total) trans 4294967383 rc 0/0 00000100:00100000:3.0:1789302887.441130:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 420 00000100:00100000:0.0:1789302887.449913:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.450021:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350534656 00000100:00100000:1.0:1789302887.450034:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350534656 00000100:00100000:1.0:1789302887.450150:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1503 00000100:00100000:1.0:1789302887.450163:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:32837:x1876219350534656:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 00010000:00010000:1.0:1789302887.450255:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.450277:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 53 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.450286:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.450342:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 53 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.450358:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 53 pid: 6919 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.450379:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371176960:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302887.450549:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3358a6c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302887.453461:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70b480 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371176960:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302887.453537:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.454566:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.454670:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350534784 00000100:00100000:3.0:1789302887.454681:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350534784 00010000:00010000:3.0:1789302887.454773:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 54 pid: 6919 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.454826:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 440 00000100:00100000:3.0:1789302887.454837:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd04050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2347:x1876219350534784:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302887.454858:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302887.454866:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 54 pid: 6919 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.454881:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 54 pid: 6919 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.454979:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fad4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2d:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f3c expref: 54 pid: 6919 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.455001:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:3.0:1789302887.455007:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.455020:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.455028:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 4 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.455137:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302887.455215:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd04050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2347:x1876219350534784:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 379us (654us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.455223:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 440 00010000:00010000:1.0:1789302887.455342:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455351:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455387:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455395:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455402:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455409:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302887.455414:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455423:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455433:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:1.0:1789302887.455437:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 21 00010000:00010000:1.0:1789302887.455442:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.455458:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095184fbbb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x21/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.455711:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32837:x1876219350534656:12345-192.168.203.11@tcp:55:lhsmtool_posix.0 Request processed in 5549us (5811us total) trans 0 rc -22/-22 00000100:00100000:1.0:1789302887.455720:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1503 00000100:00100000:2.0:1789302887.459631:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.459667:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350534912 00000100:00100000:3.0:1789302887.459682:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350534912 00000100:00100000:3.0:1789302887.459752:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 421 00000100:00100000:3.0:1789302887.459765:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350534912:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302887.459827:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.459853:0:8571:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.459863:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302887.459868:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302887.459873:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.460119:0:8571:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 11 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.460138:0:8571:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 11 pid: 6696 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.460157:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371177088:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302887.460269:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f759c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302887.465132:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.465285:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350535040 00000100:00100000:1.0:1789302887.465298:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350535040 00010000:00010000:1.0:1789302887.465352:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 12 pid: 6696 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302887.465406:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:1.0:1789302887.465417:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350535040:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302887.465436:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302887.465474:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 12 pid: 6696 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.465491:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 12 pid: 6696 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.465946:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371177088:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302887.465997:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bc5cd789/0x64fe43095184fbad lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f7b expref: 12 pid: 6696 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.466013:0:8571:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.466019:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302887.466024:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302887.466030:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.466108:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.466823:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302887.467077:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350535040:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1649us (1948us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302887.467136:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 441 00010000:00010000:3.0:1789302887.467491:0:8571:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467556:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467598:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467608:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467617:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467626:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302887.467631:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467642:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467664:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.467680:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004d843203/0x64fe43095184fbc2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.467800:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350534912:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 8037us (8377us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.467823:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 421 00000100:00100000:0.0:1789302887.481962:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.482256:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350535296 00000100:00100000:3.0:1789302887.482282:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350535296 00000100:00100000:3.0:1789302887.482930:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1504 00000100:00100000:3.0:1789302887.482960:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:32837:x1876219350535296:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302887.483202:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.483246:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.483318:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.484884:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.484939:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.484984:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.485121:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.485183:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 53 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.485215:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371177216:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302887.485550:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f334119c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302887.490373:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371177216:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302887.490416:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.493252:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.494944:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350535424 00000100:00100000:3.0:1789302887.494974:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350535424 00010000:00010000:3.0:1789302887.495100:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 54 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.495545:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 442 00000100:00100000:3.0:1789302887.495558:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2347:x1876219350535424:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302887.495584:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302887.495592:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 4/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 54 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.495609:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 54 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.495636:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fb05 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f51 expref: 54 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.495657:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:3.0:1789302887.495663:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.495675:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.495684:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.495722:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302887.495804:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2347:x1876219350535424:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 249us (2578us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.495813:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 442 00010000:00010000:0.0:1789302887.497953:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.497987:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:0.0:1789302887.498567:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968329 00000001:00080000:0.0:1789302887.498899:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968330 00000001:00080000:0.0:1789302887.498948:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968331 00000100:00100000:0.0:1789302887.499131:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371177344:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302887.499298:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302887.499418:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61840 going to sleep for 11 seconds 00000100:00100000:1.0:1789302887.500709:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371177344 00000100:00100000:1.0:1789302887.500726:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371177344 00000100:00100000:1.0:1789302887.500863:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 73 00000100:00100000:1.0:1789302887.500875:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371177344:12345-0@lo:1000:mdt00_001.0 00000100:00100000:1.0:1789302887.502376:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371177344:12345-0@lo:1000:mdt00_001.0 Request processed in 1483us (3058us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302887.502405:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 73 00000100:00100000:0.0:1789302887.503795:0:5844:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967382 gen 2 00000004:00080000:0.0:1789302887.503807:0:5844:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3fe30a80, transno 4294967382 00000100:00100000:0.0:1789302887.503877:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371177344:0@lo:1000:mdt00_001.0 00000001:00080000:0.0:1789302887.505947:0:5844:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968332 00000100:00100000:0.0:1789302887.508818:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371177472:0@lo:1000:mdt00_001.0 00000100:00100000:0.0:1789302887.508989:0:5844:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302887.509072:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302887.509332:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371177472 00000100:00100000:2.0:1789302887.509356:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371177472 00000100:00100000:2.0:1789302887.509473:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 74 00000100:00100000:2.0:1789302887.509756:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:5844:x1876219371177472:12345-0@lo:1000:mdt00_001.0 00000100:00100000:2.0:1789302887.510935:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:5844:x1876219371177472:12345-0@lo:1000:mdt00_001.0 Request processed in 1414us (1924us total) trans 4294967384 rc 0/0 00000100:00100000:2.0:1789302887.510950:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 74 00000100:00100000:0.0:1789302887.511289:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-mdtlov_UUID:5844:1876219371177472:0@lo:1000:mdt00_001.0 00010000:00010000:0.0:1789302887.511647:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511659:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511667:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511675:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.511681:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511690:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511709:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9300) 00010000:00010000:0.0:1789302887.511712:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302887.511718:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511725:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fbd0 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2d:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511766:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511774:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x50210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511780:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511786:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302887.511789:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40210400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511796:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x40218400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511805:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:0.0:1789302887.511807:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1789302887.511817:0:5844:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 3 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 106 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511829:0:5844:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 106 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511839:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 106 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302887.511893:0:5844:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 106 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302887.511911:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371177600:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302887.512027:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f334119c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302887.517488:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.517955:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46a00 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371177600:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302887.518060:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.518073:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095184f770 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 5 type: IBT flags: 0x44a19400000020 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.518240:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.518259:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 106 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.518271:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 4/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.518334:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 106 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.518353:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 106 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.518370:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371177728:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302887.518484:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599480 going to sleep for 11 seconds 00000100:00100000:3.0:1789302887.522003:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350535552 00000100:00100000:3.0:1789302887.522024:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350535552 00000100:00100000:3.0:1789302887.522114:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 323 00000100:00100000:3.0:1789302887.522127:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f076ea050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32805:x1876219350535552:12345-192.168.203.11@tcp:49:cat.0 00000100:00100000:3.0:1789302887.523411:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f076ea050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32805:x1876219350535552:12345-192.168.203.11@tcp:49:cat.0 Request processed in 1268us (5942us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.523436:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 323 00000100:00100000:0.0:1789302887.524586:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.525387:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350535680 00000100:00100000:3.0:1789302887.525409:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350535680 00010000:00010000:3.0:1789302887.525624:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 107 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302887.525666:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 443 00000100:00100000:3.0:1789302887.525696:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:2347:x1876219350535680:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302887.525715:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302887.525740:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 107 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.525831:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 107 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.525914:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:3.0:1789302887.525933:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,1 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.525948:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.525971:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 4/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302887.526027:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302887.526154:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:2347:x1876219350535680:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 477us (1589us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.526174:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 443 00010000:00010000:2.0:1789302887.529879:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe43095184f78c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x52:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0e8d expref: 106 pid: 5845 timeout: 824 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.529914:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371177728:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302887.529946:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.529957:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.529963:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.529981:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,1 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.529988:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 2/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.529997:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530005:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302887.530011:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 3/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530020:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 4/0,0 mode: EX/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530039:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302887.530042:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302887.530048:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 2/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530054:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fbd7 lrc: 0/0,0 mode: --/EX res: [0x200000401:0x52:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530067:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530074:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530080:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530086:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302887.530089:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530095:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530120:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:2.0:1789302887.530122:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302887.530126:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.530132:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fbc9 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x52:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302887.530436:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32837:x1876219350535296:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 47494us (48508us total) trans 4294968333 rc 0/0 00000100:00100000:2.0:1789302887.530463:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1504 00000020:00080000:2.0:1789302887.530471:0:5844:0:(tgt_main.c:784:tgt_ses_key_exit()) total 5 transactions per RPC 00000100:00100000:3.0:1789302887.531652:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.531790:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000040:00080000:0.0:1789302887.532527:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302887.532534:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302887.533018:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:0.0:1789302887.533028:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302887.534664:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350535808 00000100:00100000:2.0:1789302887.534680:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350535808 00000100:00100000:2.0:1789302887.534762:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 422 00000100:00100000:2.0:1789302887.534775:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+15:32805:x1876219350535808:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302887.534797:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302887.534824:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c2ee1d38/0x64fe43095184fbde lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f82 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302887.534856:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1e:0x0] initial LVB size: 158, mtime: 0x6aa69864, atime: 0x6aa69864, ctime: 0x6aa69866, blocks: 0xb 00010000:00010000:2.0:1789302887.534885:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=724s v=(0 0) 00010000:00010000:2.0:1789302887.534891:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302887.534902:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c2ee1d38/0x64fe43095184fbde lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f82 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.534922:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c2ee1d38/0x64fe43095184fbde lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->4194303) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f82 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.534937:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030400, rc 0) 00000100:00100000:2.0:1789302887.535004:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:32805:x1876219350535808:12345-192.168.203.11@tcp:101:cat.0 Request processed in 232us (3358us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302887.535012:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 422 00000100:00100000:0.0:1789302887.535076:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.536004:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350536320 00000100:00100000:2.0:1789302887.536015:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350536320 00000100:00100000:2.0:1789302887.536059:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 324 00000100:00100000:2.0:1789302887.536068:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:32837:x1876219350536320:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000004:00080000:2.0:1789302887.537190:0:5846:0:(mdd_object.c:4014:mdd_close()) Object [0x200000402:0x2d:0x0] is deleted from orphan list, OSS objects to be destroyed. 00000001:00080000:2.0:1789302887.537334:0:5846:0:(tgt_lastrcvd.c:2117:tgt_txn_stop_cb()) More than one transaction 4294968334 00010000:00010000:2.0:1789302887.537397:0:5846:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537416:0:5846:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537431:0:5846:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537442:0:5846:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010080000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537457:0:5846:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537463:0:5846:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x50010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537487:0:5846:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1789302887.537633:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:32837:x1876219350536320:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 1563us (3130us total) trans 4294968335 rc 0/0 00000100:00100000:2.0:1789302887.537644:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 324 00000020:00080000:2.0:1789302887.537650:0:5846:0:(tgt_main.c:784:tgt_ses_key_exit()) total 2 transactions per RPC 00010000:00010000:2.0:1789302887.537685:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302887.537694:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537701:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302887.537705:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537714:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537728:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2d:0x0].0x0 (ffff960f124a9a00) 00010000:00010000:2.0:1789302887.537731:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789302887.537736:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537742:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbe5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2d:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5846 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.537761:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302887.537763:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302887.537764:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302887.538669:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350536064 00000100:00100000:1.0:1789302887.538680:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350536064 00000100:00100000:1.0:1789302887.538748:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 70 00000100:00100000:1.0:1789302887.538759:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32806:x1876219350536064:12345-192.168.203.11@tcp:103:cat.0 00010000:00010000:1.0:1789302887.538776:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302887.538799:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095184f7d9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ea2 expref: 106 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302887.538818:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f09ae1c00) 00010000:00010000:1.0:1789302887.538829:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302887.538872:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32806:x1876219350536064:12345-192.168.203.11@tcp:103:cat.0 Request processed in 115us (7081us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302887.538879:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 70 00000100:00080000:3.0:1789302887.539076:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302887.543165:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.543250:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350536448 00000100:00100000:3.0:1789302887.543274:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350536448 00000100:00100000:3.0:1789302887.543348:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1505 00000100:00100000:3.0:1789302887.543361:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32806:x1876219350536448:12345-192.168.203.11@tcp:58:cat.0 00000100:00100000:3.0:1789302887.543828:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32806:x1876219350536448:12345-192.168.203.11@tcp:58:cat.0 Request processed in 468us (676us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302887.543837:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1505 00000100:00100000:1.0:1789302887.546225:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302887.546367:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350536704 00000100:00100000:0.0:1789302887.546382:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350536704 00000100:00100000:0.0:1789302887.546466:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1506 00000100:00100000:0.0:1789302887.546479:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32809:x1876219350536704:12345-192.168.203.11@tcp:58:cat.0 00000100:00100000:0.0:1789302887.546699:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32809:x1876219350536704:12345-192.168.203.11@tcp:58:cat.0 Request processed in 221us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302887.546708:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1506 00000100:00100000:0.0:1789302887.549131:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302887.549232:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350536832 00000100:00100000:2.0:1789302887.549244:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350536832 00000100:00100000:2.0:1789302887.549324:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1507 00000100:00100000:2.0:1789302887.549337:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+105:32809:x1876219350536832:12345-192.168.203.11@tcp:101:cat.0 00010000:00010000:2.0:1789302887.549364:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302887.549393:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbec lrc: 2/0,0 mode: --/CR res: [0x200000401:0x52:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f89 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.549478:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbec lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f89 expref: 106 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.549495:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030600/0x64fe43095184fbec lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x52:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f89 expref: 106 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302887.549562:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x52:0x0].0x0 (ffff960f124a9a00) 00010000:00010000:2.0:1789302887.549566:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1789302887.549570:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030600, rc 0) 00000100:00100000:2.0:1789302887.549635:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32809:x1876219350536832:12345-192.168.203.11@tcp:101:cat.0 Request processed in 300us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302887.549643:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1507 00000100:00100000:3.0:1789302887.552607:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302887.552757:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350536960 00000100:00100000:1.0:1789302887.552771:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350536960 00002000:00100000:1.0:1789302887.552834:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1e:0x0] (0->4095) req@ffff960f3fe32300 x1876219350536960/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:463/0 lens 488/0 e 0 to 0 dl 1789302898 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302887.552846:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fe32300 with x1876219350536960 ext(0->4095) 00002000:00010000:1.0:1789302887.552852:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fe32300 00000100:00100000:1.0:1789302887.552890:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 127 00000100:00100000:1.0:1789302887.552902:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350536960:12345-192.168.203.11@tcp:3:cat.0 00000100:00100000:1.0:1789302887.553158:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350536960:12345-192.168.203.11@tcp:3:cat.0 Request processed in 257us (563us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302887.553166:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 127 00002000:00100000:1.0:1789302887.553173:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1e:0x0] (0->4095) req@ffff960f3fe32300 x1876219350536960/t0(0) o3->662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa@192.168.203.11@tcp:463/0 lens 488/4536 e 0 to 0 dl 1789302898 ref 1 fl Complete:/600/0 rc 4096/4096 job:'cat.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302887.553182:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fe32300 with x1876219350536960 ext(0->4095) 00002000:00010000:1.0:1789302887.553186:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fe32300 00000100:00100000:1.0:1789302887.613984:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302887.614413:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350537216 00000100:00100000:3.0:1789302887.614428:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350537216 00000100:00100000:2.0:1789302887.614653:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 325 00000100:00100000:2.0:1789302887.614669:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:32835:x1876219350537216:12345-192.168.203.11@tcp:35:cat.0 00000100:00100000:2.0:1789302887.615122:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:32835:x1876219350537216:12345-192.168.203.11@tcp:35:cat.0 Request processed in 456us (1154us total) trans 4294968336 rc 0/0 00000100:00100000:2.0:1789302887.615132:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 325 00010000:00010000:1.0:1789302888.076663:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302888.461554:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371177984:0@lo:400:kworker.0 00000100:00100000:0.0:1789302888.461650:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371178112:0@lo:400:kworker.0 00000100:00100000:3.0:1789302888.461667:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371178240:0@lo:400:kworker.0 00000100:00100000:2.0:1789302888.461740:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371177856:0@lo:400:kworker.0 00000100:00100000:0.0:1789302888.461777:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302888.464723:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302888.466427:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371178112 00000100:00100000:3.0:1789302888.466457:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371178112 00000100:00080000:1.0:1789302888.467025:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302888.467043:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371178240 00000100:00100000:1.0:1789302888.467049:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371178240 00000100:00100000:1.0:1789302888.467101:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:1.0:1789302888.467111:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371178240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302888.467200:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371178240:12345-0@lo:400:kworker.0 Request processed in 90us (2487us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302888.467207:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:3.0:1789302888.467999:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:3.0:1789302888.468014:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371178112:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302888.468075:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371178112:12345-0@lo:400:kworker.0 Request processed in 67us (6299us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302888.468082:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:3.0:1789302888.468186:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371178240:0@lo:400:kworker.0 00000100:00100000:2.0:1789302888.469455:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302888.469523:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371177856 00000100:00100000:2.0:1789302888.469534:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371177856 00000100:00100000:2.0:1789302888.469685:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:2.0:1789302888.469707:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371177856:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302888.469766:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371177856:12345-0@lo:400:kworker.0 Request processed in 71us (315us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302888.469774:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:3.0:1789302888.470095:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371177856:0@lo:400:kworker.0 00000100:00100000:3.0:1789302888.470110:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371178368:0@lo:400:kworker.0 00000100:00100000:0.0:1789302888.477242:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371178112:0@lo:400:kworker.0 00000100:00100000:0.0:1789302888.477326:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302888.477397:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371178368 00000100:00100000:0.0:1789302888.477421:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371178368 00000100:00100000:0.0:1789302888.477577:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:1.0:1789302888.477796:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302888.477836:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371178496:0@lo:400:kworker.0 00000100:00100000:1.0:1789302888.478354:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302888.478702:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371178496 00000100:00100000:3.0:1789302888.478711:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371178496 00000100:00100000:3.0:1789302888.478762:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 531 00000100:00100000:3.0:1789302888.478771:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371178496:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302888.478848:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371178496:12345-0@lo:400:kworker.0 Request processed in 78us (494us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302888.478856:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 531 00000100:00100000:1.0:1789302888.479422:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371178496:0@lo:400:kworker.0 00000100:00100000:0.0:1789302888.482371:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371178368:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302888.484761:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371177984 00000100:00100000:2.0:1789302888.484794:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371177984 00000100:00100000:2.0:1789302888.484904:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:2.0:1789302888.484926:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371177984:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302888.489902:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f708380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371177984:12345-0@lo:400:kworker.0 Request processed in 4960us (12077us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302888.489938:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00100000:2.0:1789302888.495683:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371177984:0@lo:400:kworker.0 00000100:00100000:0.0:1789302888.496787:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371178368:12345-0@lo:400:kworker.0 Request processed in 14485us (19450us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302888.496806:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00080000:0.0:1789302888.496857:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302888.496865:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302888.497009:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371178368:0@lo:400:kworker.0 00000001:00080000:1.0:1789302888.569551:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967380 is committed 00000001:00080000:1.0:1789302888.569590:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967381 is committed 00000040:00080000:1.0:1789302888.589558:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302888.589574:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302888.590399:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 4 00000040:00080000:1.0:1789302888.590418:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302888.975074:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302888.975234:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371178624:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302888.975394:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302888.975472:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371178624 00000100:00100000:3.0:1789302888.975483:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371178624 00000100:00100000:3.0:1789302888.975567:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1789302888.975578:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371178624:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302888.975623:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302888.975679:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371178624:12345-0@lo:13:osp-pre-1-0.0 Request processed in 103us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302888.975687:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:0.0:1789302888.975984:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967381 gen 2 00000004:00080000:0.0:1789302888.976002:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f63e300, transno 4294967381 00000100:00100000:0.0:1789302888.976043:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371178624:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302889.108300:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302889.491095:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371178752:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302889.491199:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302889.491414:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371178752 00000100:00100000:1.0:1789302889.491426:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371178752 00000100:00100000:1.0:1789302889.491540:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 219 00000100:00100000:1.0:1789302889.491554:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371178752:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302889.491601:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3858759680 avail 3856662528 00000100:00080000:2.0:1789302889.491700:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302889.495444:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371178752:12345-0@lo:13:osp-pre-0-0.0 Request processed in 3870us (4220us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302889.495463:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 219 00000100:00100000:3.0:1789302889.495930:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371178752:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302889.742993:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302889.743051:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302889.743077:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302889.743120:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302889.743138:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302889.743161:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302889.743190:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302889.743226:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350537856 00000100:00100000:3.0:1789302889.743239:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350537856 00000100:00100000:3.0:1789302889.743311:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350537472 00000100:00100000:3.0:1789302889.743314:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350537472 00000100:00100000:3.0:1789302889.743348:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00100000:3.0:1789302889.743361:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:2346:x1876219350537856:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302889.743482:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:2346:x1876219350537856:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 123us (403us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302889.743492:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00100000:3.0:1789302889.743859:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 511 00000100:00100000:3.0:1789302889.743870:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:2346:x1876219350537472:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302889.743911:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:2346:x1876219350537472:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 45us (771us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302889.743917:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 511 00000100:00100000:3.0:1789302889.743940:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350537600 00000100:00100000:3.0:1789302889.743945:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350537600 00000100:00100000:3.0:1789302889.743976:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350538112 00000100:00100000:3.0:1789302889.743979:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350538112 00000100:00100000:3.0:1789302889.743997:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350537728 00000100:00100000:3.0:1789302889.744000:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350537728 00000100:00100000:3.0:1789302889.744016:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350537984 00000100:00100000:3.0:1789302889.744018:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350537984 00000100:00100000:3.0:1789302889.744033:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 532 00000100:00100000:3.0:1789302889.744040:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350537600:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302889.744068:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350537600:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (1017us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302889.744079:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 532 00000100:00100000:3.0:1789302889.745028:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00100000:3.0:1789302889.745046:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350538112:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302889.745110:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350538112:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 71us (1988us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302889.745117:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00100000:3.0:1789302889.745129:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 534 00000100:00100000:3.0:1789302889.745136:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350537728:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302889.745161:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350537728:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (2000us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302889.745168:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 534 00000100:00100000:3.0:1789302889.745179:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 535 00000100:00100000:3.0:1789302889.745186:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350537984:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302889.745215:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350537984:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (2024us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302889.745221:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 535 00000100:00100000:2.0:1789302889.745765:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350537344 00000100:00100000:2.0:1789302889.745776:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350537344 00000100:00100000:2.0:1789302889.745843:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 622 00000100:00100000:2.0:1789302889.745851:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350537344:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302889.745896:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350537344:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 46us (2918us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302889.745903:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 622 00000100:00080000:2.0:1789302889.745953:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302889.745958:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302889.745962:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302890.124683:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302890.637089:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302890.637099:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302890.637352:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 45 count 3 00000040:00080000:0.0:1789302890.637359:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302891.149782:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302892.172763:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1789302892.673616:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967384 is committed 00000001:00080000:1.0:1789302892.673637:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967383 is committed 00000001:00080000:1.0:1789302892.719084:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968333 is committed 00000020:00080000:1.0:1789302892.719099:0:5865:0:(update_trans.c:59:top_multiple_thandle_dump()) lustre-MDT0000-osd tmt ffff960f0cf61840 refcount 2 committed 0 result 0 batchid 35 00000020:00080000:1.0:1789302892.719105:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f05039000 obd lustre-OST0000-osc-MDT0000 committed 1 started 1 stopped 1 result 0 sub_th ffff960f0cf61780 00000020:00080000:1.0:1789302892.719108:0:5865:0:(update_trans.c:67:top_multiple_thandle_dump()) st ffff960f05039c00 obd lustre-MDT0000-osd committed 1 started 1 stopped 1 result 0 sub_th ffff960f3b080800 00000020:00080000:3.0:1789302892.719163:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 35 committed batchid 34 00000001:00080000:1.0:1789302892.719282:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968336 is committed 00000001:00080000:1.0:1789302892.719288:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968335 is committed 00000020:00080000:3.0:1789302892.719367:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 34 00000100:00100000:3.0:1789302893.121906:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302893.123050:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350538240 00000100:00100000:0.0:1789302893.123063:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350538240 00000100:00100000:0.0:1789302893.123158:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1508 00000100:00100000:0.0:1789302893.123171:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32757:x1876219350538240:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 00000100:00100000:0.0:1789302893.123298:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:32757:x1876219350538240:12345-192.168.203.11@tcp:60:lhsmtool_posix.0 Request processed in 131us (1401us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302893.123306:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1508 00000100:00080000:2.0:1789302893.124312:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302893.196955:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302893.580860:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371179136:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.580933:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302893.581001:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1789302893.581007:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371179008:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581030:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302893.581052:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371179520:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581078:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302893.581119:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371178880:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581124:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371179136 00000100:00100000:0.0:1789302893.581134:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371179136 00000100:00100000:2.0:1789302893.581158:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302893.581187:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371179392:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581195:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371179008 00000100:00100000:0.0:1789302893.581200:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371179008 00000100:00100000:0.0:1789302893.581224:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:0.0:1789302893.581232:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371179136:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302893.581281:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302893.581292:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371179136:12345-0@lo:400:kworker.0 Request processed in 58us (357us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302893.581300:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:2.0:1789302893.581306:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1789302893.581314:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371179264:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581317:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371179392 00000100:00100000:0.0:1789302893.581322:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371179392 00000100:00100000:2.0:1789302893.581339:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302893.581345:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:0.0:1789302893.581354:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371179008:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302893.581366:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371178880 00000100:00100000:2.0:1789302893.581371:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371178880 00000100:00100000:0.0:1789302893.581392:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3301:x1876219371179008:12345-0@lo:400:kworker.0 Request processed in 39us (360us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302893.581401:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:2.0:1789302893.581415:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:2.0:1789302893.581425:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371178880:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581433:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 514 00000100:00100000:0.0:1789302893.581443:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371179392:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302893.581464:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371178880:12345-0@lo:400:kworker.0 Request processed in 40us (305us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302893.581472:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00000100:00100000:0.0:1789302893.581485:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371179392:12345-0@lo:400:kworker.0 Request processed in 43us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302893.581493:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 514 00000100:00080000:2.0:1789302893.581529:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302893.581557:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371178880:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581558:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3301:1876219371179008:0@lo:400:kworker.0 00000100:00100000:2.0:1789302893.581576:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371179392:0@lo:400:kworker.0 00000100:00100000:0.0:1789302893.581580:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371179136:0@lo:400:kworker.0 00000100:00100000:3.0:1789302893.581705:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371179520 00000100:00100000:3.0:1789302893.581710:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371179520 00000100:00100000:3.0:1789302893.581744:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371179264 00000100:00100000:3.0:1789302893.581747:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371179264 00000100:00100000:3.0:1789302893.581783:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:3.0:1789302893.581794:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371179520:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302893.581824:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 537 00000100:00100000:2.0:1789302893.581833:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371179264:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302893.581870:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371179264:12345-0@lo:400:kworker.0 Request processed in 39us (532us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302893.581876:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 537 00000100:00100000:2.0:1789302893.581902:0:3303:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967384 gen 2 00000004:00080000:2.0:1789302893.581907:0:3303:0:(osp_trans.c:1040:osp_request_commit_cb()) trans no 4294967384 committed transno 4294967384 00000100:00100000:2.0:1789302893.581925:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371179264:0@lo:400:kworker.0 00000100:00080000:2.0:1789302893.581955:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302893.581959:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302893.588232:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371179520:12345-0@lo:400:kworker.0 Request processed in 6431us (7143us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302893.588247:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:1.0:1789302893.594588:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371179520:0@lo:400:kworker.0 00000100:00100000:1.0:1789302894.093290:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371179648:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302894.093354:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302894.093486:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371179648 00000100:00100000:0.0:1789302894.093493:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371179648 00000100:00100000:0.0:1789302894.093553:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:0.0:1789302894.093561:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371179648:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302894.093597:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302894.093645:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371179648:12345-0@lo:13:osp-pre-1-0.0 Request processed in 84us (290us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302894.093651:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:0.0:1789302894.093701:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371179648:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302894.220705:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302894.604871:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371179776:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302894.604936:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302894.605158:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371179776 00000100:00100000:0.0:1789302894.605166:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371179776 00000100:00100000:0.0:1789302894.605214:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 221 00000100:00100000:0.0:1789302894.605223:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371179776:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302894.605255:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 90112000 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302894.605342:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ad880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371179776:12345-0@lo:13:osp-pre-0-0.0 Request processed in 119us (404us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302894.605350:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 221 00000100:00100000:3.0:1789302894.608189:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371179776:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302894.884727:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302894.884768:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302894.884792:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302894.884811:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302894.884828:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302894.884843:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302894.884862:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302894.884885:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350538624 00000100:00100000:2.0:1789302894.884893:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350538624 00000100:00100000:2.0:1789302894.884940:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350539136 00000100:00100000:2.0:1789302894.884942:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350539136 00000100:00100000:2.0:1789302894.884957:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350538752 00000100:00100000:2.0:1789302894.884959:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350538752 00000100:00100000:2.0:1789302894.884982:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350539008 00000100:00100000:2.0:1789302894.884984:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350539008 00000100:00100000:2.0:1789302894.884999:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 538 00000100:00100000:2.0:1789302894.885007:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350538624:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302894.885068:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350538624:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 61us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302894.885075:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 538 00000100:00100000:2.0:1789302894.885088:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 539 00000100:00100000:2.0:1789302894.885094:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350539136:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302894.885113:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350539136:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (301us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302894.885120:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 539 00000100:00100000:2.0:1789302894.885129:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 540 00000100:00100000:2.0:1789302894.885134:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350538752:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302894.885151:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350538752:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 17us (306us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302894.885157:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 540 00000100:00100000:2.0:1789302894.885165:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 541 00000100:00100000:2.0:1789302894.885171:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350539008:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302894.885187:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350539008:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 17us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302894.885192:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 541 00000100:00100000:2.0:1789302894.885206:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350538880 00000100:00100000:2.0:1789302894.885209:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350538880 00000100:00100000:2.0:1789302894.885228:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350538496 00000100:00100000:2.0:1789302894.885230:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350538496 00000100:00100000:2.0:1789302894.885246:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 515 00000100:00100000:2.0:1789302894.885252:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2347:x1876219350538880:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302894.885274:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2347:x1876219350538880:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (481us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302894.885280:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 515 00000100:00100000:2.0:1789302894.885289:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 516 00000100:00100000:2.0:1789302894.885294:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350538496:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302894.885311:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:2347:x1876219350538496:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 17us (483us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302894.885317:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 516 00000100:00080000:2.0:1789302894.885342:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302894.885345:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302894.885347:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302894.886082:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350538368 00000100:00100000:0.0:1789302894.886089:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350538368 00000100:00100000:0.0:1789302894.886117:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 624 00000100:00100000:0.0:1789302894.886123:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350538368:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302894.886151:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350538368:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (1434us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302894.886157:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 624 00000001:02000400:1.0:1789302895.102428:0:22689:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-hsm test 12u: Multiple reads on multiple HSM imported files in parallel ========================================================== 08:34:53 (1789302893) 00010000:00010000:1.0:1789302895.247288:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302896.136356:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.136473:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350539264 00000100:00100000:3.0:1789302896.136484:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350539264 00000100:00100000:3.0:1789302896.136604:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1509 00000100:00100000:3.0:1789302896.136615:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:33445:x1876219350539264:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 00000100:00100000:3.0:1789302896.136717:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f709880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:33445:x1876219350539264:12345-192.168.203.11@tcp:59:lhsmtool_posix.0 Request processed in 103us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302896.136724:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1509 00000100:00080000:2.0:1789302896.136834:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302896.202620:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.202983:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350539392 00000100:00100000:2.0:1789302896.202997:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350539392 00000100:00080000:0.0:1789302896.203087:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302896.203095:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1510 00000100:00100000:2.0:1789302896.203107:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:33446:x1876219350539392:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302896.203127:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.203147:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fbf3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f90 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203207:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203221:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203235:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203249:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203256:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203262:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203483:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203491:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203498:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203558:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.203563:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203571:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203583:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:2.0:1789302896.203585:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.203590:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203595:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fbfa lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203609:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203615:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203621:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203626:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.203628:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203634:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203640:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff960f124a9500) 00010000:00010000:2.0:1789302896.203642:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.203645:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203649:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc01 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203676:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fbf3 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f90 expref: 107 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203686:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:2.0:1789302896.203688:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.203691:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fbf3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f90 expref: 107 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.203699:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030200, rc 0) 00000100:00100000:2.0:1789302896.203769:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:33446:x1876219350539392:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 664us (1162us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302896.203776:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1510 00000100:00100000:1.0:1789302896.224318:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.225678:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350539648 00000100:00100000:0.0:1789302896.225697:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350539648 00000100:00100000:0.0:1789302896.225778:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1511 00000100:00100000:0.0:1789302896.225791:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:33446:x1876219350539648:12345-192.168.203.11@tcp:36:lfs.0 00010000:00010000:0.0:1789302896.225881:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302896.225916:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe43095184f516 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dbb expref: 106 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.225941:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302896.225948:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302896.226233:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.226250:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 49 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.226260:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.226303:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 49 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.226321:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 49 pid: 18427 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302896.226339:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371179904:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302896.226494:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302896.231420:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371179904:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302896.231465:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302896.241061:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.241254:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350539776 00000100:00100000:3.0:1789302896.241269:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350539776 00010000:00010000:3.0:1789302896.241323:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 50 pid: 18427 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302896.241380:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 444 00000100:00100000:3.0:1789302896.241402:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:2347:x1876219350539776:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302896.241423:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302896.241432:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 50 pid: 18427 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.241454:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 50 pid: 18427 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.241488:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184f555 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0dd0 expref: 50 pid: 18427 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.241557:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:3.0:1789302896.241565:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.241575:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.241583:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 4/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.241611:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302896.241891:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:2347:x1876219350539776:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 491us (854us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302896.241901:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 444 00010000:00010000:0.0:1789302896.243069:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.243089:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.243122:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.243136:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.243150:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302896.248930:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00010000:00010000:0.0:1789302896.248959:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.248972:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.248981:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.248988:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.248995:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.249005:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.249025:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f10d3c000) 00010000:00010000:0.0:1789302896.249029:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.249046:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.249054:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc08 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302896.249082:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000004205e147 reply state 000000008d136f1d transno 4294968337 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302896.249094:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302896.249230:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:33446:x1876219350539648:12345-192.168.203.11@tcp:36:lfs.0 Request processed in 23435us (24921us total) trans 4294968337 rc 0/0 00000100:00100000:0.0:1789302896.249240:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1511 00010000:00010000:3.0:1789302896.264932:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.264954:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 2/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.264963:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.264975:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302896.264980:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 3/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.264988:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 4/0,0 mode: PW/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.265007:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff960f09ae1800) 00010000:00010000:3.0:1789302896.265011:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.265016:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 2/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.265022:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc0f lrc: 0/0,0 mode: --/PW res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.278046:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302896.286311:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.286828:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350539904 00000100:00100000:0.0:1789302896.286842:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350539904 00000100:00100000:0.0:1789302896.286917:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1512 00000100:00100000:0.0:1789302896.286929:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+106:33447:x1876219350539904:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302896.286960:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.286992:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc16 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f97 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.287087:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc1d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.287104:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.287118:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302896.287163:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:0.0:1789302896.287169:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302896.287194:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc1d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.287223:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc16 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f97 expref: 108 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.287246:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b600/0x64fe43095184fc1d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f97 expref: 107 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.287256:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f09ae1200) 00010000:00010000:0.0:1789302896.287259:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302896.287264:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b600, rc 0) 00000100:00100000:0.0:1789302896.287343:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:33447:x1876219350539904:12345-192.168.203.11@tcp:101:bash.0 Request processed in 417us (1047us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302896.287351:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1512 00000100:00100000:1.0:1789302896.298901:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.299584:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540032 00000100:00100000:2.0:1789302896.299599:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540032 00000100:00100000:2.0:1789302896.299692:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1513 00000100:00100000:2.0:1789302896.299706:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+107:33447:x1876219350540032:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302896.299734:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.299761:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fc24 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0f9e expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300155:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300174:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300187:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300203:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300210:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300218:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300335:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184fc39 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300344:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184fc39 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300356:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184fc39 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302896.300402:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:2.0:1789302896.300407:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302896.300429:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184fc39 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300444:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300451:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300468:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300475:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.300480:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300489:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300717:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f09ae1200) 00010000:00010000:2.0:1789302896.300722:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.300728:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300735:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc2b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300756:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300762:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300767:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300773:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.300776:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300781:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.300789:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff960f124a9500) 00010000:00010000:2.0:1789302896.300791:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.300794:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.301793:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc32 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.301864:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fc24 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f9e expref: 109 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.301879:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031400/0x64fe43095184fc39 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0f9e expref: 108 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.301902:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.301905:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302896.301908:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031400, rc 0) 00000100:00100000:2.0:1789302896.302011:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:33447:x1876219350540032:12345-192.168.203.11@tcp:101:bash.0 Request processed in 2307us (3134us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302896.302021:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1513 00000100:00100000:1.0:1789302896.316197:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.316717:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540160 00000100:00100000:0.0:1789302896.316732:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540160 00000100:00100000:0.0:1789302896.316818:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1514 00000100:00100000:0.0:1789302896.316831:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:33447:x1876219350540160:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302896.316860:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.316900:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc40 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fa5 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.316992:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317010:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317029:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317048:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317057:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317065:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317141:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317150:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317158:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317165:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.317170:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317178:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317193:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.317196:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.317202:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317209:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fc47 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317222:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317228:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317234:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317240:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.317243:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317250:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317258:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f09ae1f00) 00010000:00010000:0.0:1789302896.317261:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.317264:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317269:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fc4e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317308:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc40 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fa5 expref: 109 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317327:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.317329:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.317333:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fc40 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fa5 expref: 109 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.317344:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302896.317440:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:33447:x1876219350540160:12345-192.168.203.11@tcp:101:bash.0 Request processed in 611us (1256us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302896.317449:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1514 00000100:00100000:1.0:1789302896.327996:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.328270:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540288 00000100:00100000:3.0:1789302896.328284:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540288 00000100:00100000:3.0:1789302896.328364:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1515 00000100:00100000:3.0:1789302896.328378:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+49:33448:x1876219350540288:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302896.328414:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302896.328446:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fc55 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fac expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.330153:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fc5c lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.330180:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fc5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.330200:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fc5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302896.330248:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 16384 00000004:00200000:3.0:1789302896.330254:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302896.330279:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fc5c lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.330314:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fc55 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fac expref: 51 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.330339:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0200/0x64fe43095184fc5c lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fac expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.330349:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f09ae1200) 00010000:00010000:3.0:1789302896.330353:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302896.330358:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0200, rc 0) 00000100:00100000:3.0:1789302896.330603:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:33448:x1876219350540288:12345-192.168.203.11@tcp:101:rm.0 Request processed in 2225us (2620us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302896.330613:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1515 00000100:00100000:1.0:1789302896.342641:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.343172:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540416 00000100:00100000:2.0:1789302896.343185:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540416 00000100:00100000:2.0:1789302896.343262:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1516 00000100:00100000:2.0:1789302896.343274:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+50:33448:x1876219350540416:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302896.343300:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.343327:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fc63 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fb3 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.343425:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.343444:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.343784:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344096:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344108:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344122:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344237:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184fc78 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344246:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184fc78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344254:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184fc78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302896.344289:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:2.0:1789302896.344294:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302896.344317:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184fc78 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344331:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344338:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344345:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344352:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.344356:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344365:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344379:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (ffff960f09ae1200) 00010000:00010000:2.0:1789302896.344383:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.344401:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344408:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fc6a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344423:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344430:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344436:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344441:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.344443:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344449:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344458:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x3b8202c4 (ffff960f124a9500) 00010000:00010000:2.0:1789302896.344460:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.344463:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.344477:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fc71 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x3b8202c4 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.345104:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fc63 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fb3 expref: 52 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.345127:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031800/0x64fe43095184fc78 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fb3 expref: 51 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.345135:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.345137:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:2.0:1789302896.345141:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031800, rc 0) 00000100:00100000:2.0:1789302896.345241:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33448:x1876219350540416:12345-192.168.203.11@tcp:101:rm.0 Request processed in 1968us (2613us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302896.345252:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1516 00000100:00100000:1.0:1789302896.353301:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.353402:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540544 00000100:00100000:2.0:1789302896.353416:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540544 00000100:00100000:2.0:1789302896.353498:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1517 00000100:00100000:2.0:1789302896.353543:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33448:x1876219350540544:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302896.353572:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.353601:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fc7f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fba expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353712:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353730:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353744:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353761:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353769:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353776:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353850:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353858:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353866:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353873:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.353878:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353897:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353911:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.353915:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.353920:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353927:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fc86 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353941:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353948:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353954:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353959:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.353962:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353967:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353973:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f124a9b00) 00010000:00010000:2.0:1789302896.353975:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.353978:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.353984:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fc8d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.354018:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fc7f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fba expref: 52 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.354029:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.354031:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.354035:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fc7f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fba expref: 52 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.354047:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031c00, rc 0) 00000100:00100000:2.0:1789302896.354108:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33448:x1876219350540544:12345-192.168.203.11@tcp:101:rm.0 Request processed in 574us (822us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302896.354115:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1517 00000100:00100000:1.0:1789302896.362122:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.362279:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540672 00000100:00100000:3.0:1789302896.362292:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540672 00000100:00100000:3.0:1789302896.362357:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1518 00000100:00100000:3.0:1789302896.362368:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33448:x1876219350540672:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302896.362405:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302896.362431:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fc94 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fc1 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.362755:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.362773:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.362787:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.362878:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363022:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363032:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363104:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363112:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363120:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363126:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302896.363131:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363145:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363160:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302896.363162:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.363169:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363175:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fc9b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363191:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363197:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363204:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363209:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302896.363211:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363218:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363225:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302896.363228:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.363230:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363240:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fca2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363279:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fc94 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fc1 expref: 52 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363290:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302896.363292:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.363296:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fc94 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fc1 expref: 52 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.363308:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302896.363401:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33448:x1876219350540672:12345-192.168.203.11@tcp:101:rm.0 Request processed in 1033us (1293us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302896.363409:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1518 00000100:00100000:1.0:1789302896.430211:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.430314:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540800 00000100:00100000:2.0:1789302896.430327:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540800 00000100:00100000:2.0:1789302896.430413:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1519 00000100:00100000:2.0:1789302896.430426:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33449:x1876219350540800:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302896.430452:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.430480:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fca9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fc8 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430617:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430636:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430659:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430675:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430682:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430688:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430757:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430765:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430773:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430780:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.430785:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430794:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430819:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.430822:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.430828:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430834:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fcb0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430849:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430856:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430862:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430868:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7f800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.430872:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.430878:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.431167:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f124a9b00) 00010000:00010000:2.0:1789302896.431171:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.431175:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.431183:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fcb7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.431229:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fca9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fc8 expref: 52 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.431241:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.431243:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.431247:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095184fca9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fc8 expref: 52 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.431257:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7fc00, rc 0) 00000100:00100000:2.0:1789302896.431332:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33449:x1876219350540800:12345-192.168.203.11@tcp:101:cp.0 Request processed in 908us (1133us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302896.431341:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1519 00000100:00100000:1.0:1789302896.448470:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.448692:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350540928 00000100:00100000:0.0:1789302896.448706:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350540928 00000100:00100000:0.0:1789302896.448778:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1520 00000100:00100000:0.0:1789302896.448790:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33449:x1876219350540928:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302896.449112:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.449141:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fcbe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fcf expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449227:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449245:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449259:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449280:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449288:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449295:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449359:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449367:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449374:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449381:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.449398:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449407:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449423:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.449426:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.449433:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449440:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fcc5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449456:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449463:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449470:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449475:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.449478:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449484:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449492:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f09ae1f00) 00010000:00010000:0.0:1789302896.449494:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.449498:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449683:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fccc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449728:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fcbe lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fcf expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449741:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.449744:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.449749:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fcbe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fcf expref: 52 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.449767:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302896.449867:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33449:x1876219350540928:12345-192.168.203.11@tcp:101:cp.0 Request processed in 1079us (1422us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302896.449877:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1520 00000100:00100000:1.0:1789302896.454282:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.455730:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350541184 00000100:00100000:3.0:1789302896.455746:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350541184 00000100:00100000:3.0:1789302896.455843:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1521 00000100:00100000:3.0:1789302896.455856:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+51:33449:x1876219350541184:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302896.456144:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302896.456174:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fcd3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fd6 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.456337:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.456357:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.456371:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.456397:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.456405:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.456412:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302896.458703:0:5845:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1e:0x0], index 1: 0 00010000:00010000:3.0:1789302896.458988:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459007:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459021:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459359:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459369:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459376:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459391:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302896.459398:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459407:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459423:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302896.459426:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.459432:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459440:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fcda lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302896.459638:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005b6d376a reply state 000000009dd755ad transno 4294968338 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302896.459703:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302896.459754:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fcd3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 54 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459771:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 53 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.459779:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302896.459781:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302896.459784:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0c00, rc 0) 00000100:00100000:3.0:1789302896.459990:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33449:x1876219350541184:12345-192.168.203.11@tcp:101:cp.0 Request processed in 4134us (6100us total) trans 4294968338 rc 0/0 00000100:00100000:3.0:1789302896.460001:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1521 00010000:00010000:2.0:1789302896.471352:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.471378:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.471398:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.471405:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.471412:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.471421:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.471444:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f0cdf2600) 00010000:00010000:2.0:1789302896.471448:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.471454:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.471461:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fce1 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302896.493259:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.494322:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350541312 00000100:00100000:2.0:1789302896.494342:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350541312 00000100:00100000:2.0:1789302896.494460:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 423 00000100:00100000:2.0:1789302896.494481:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350541312:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302896.495026:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00080000:0.0:1789302896.495193:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1789302896.504569:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fdd expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302896.504618:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1e:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:3.0:1789302896.504955:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=733s v=(0 0) 00010000:00010000:3.0:1789302896.504962:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302896.504978:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.505005:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.505023:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302896.505133:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350541312:12345-192.168.203.11@tcp:101:cp.0 Request processed in 10654us (11888us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302896.505144:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 423 00000100:00100000:3.0:1789302896.508997:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.509162:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350541440 00000100:00100000:2.0:1789302896.509174:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350541440 00000100:00100000:2.0:1789302896.509247:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 424 00000100:00100000:2.0:1789302896.509260:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:33449:x1876219350541440:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302896.509283:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.509308:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0fe4 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.509350:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.509360:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302896.509365:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302896.509371:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.509426:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 11 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.509443:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 11 pid: 6697 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302896.509470:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371180032:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302896.509777:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b0c0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302896.520316:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:18578:1876219371180032:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302896.520370:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.520414:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7fc00, rc 0) 00000100:00100000:2.0:1789302896.520496:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:33449:x1876219350541440:12345-192.168.203.11@tcp:101:cp.0 Request processed in 11237us (11513us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302896.520552:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 424 00000100:00100000:3.0:1789302896.521768:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.523942:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350541568 00000100:00100000:2.0:1789302896.523962:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350541568 00010000:00010000:2.0:1789302896.524021:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 11 pid: 6697 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302896.524082:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 445 00000100:00100000:2.0:1789302896.524093:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2345:x1876219350541568:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302896.524113:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302896.524144:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 11 pid: 6697 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.524161:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 11 pid: 6697 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.524191:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184fcf6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fdd expref: 11 pid: 6697 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.524213:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302896.524218:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302896.524229:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.524243:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.524290:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 10 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302896.524317:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371180160:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302896.532623:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302896.553392:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.553489:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350541824 00000100:00100000:0.0:1789302896.553541:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350541824 00000100:00100000:0.0:1789302896.553615:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:0.0:1789302896.553627:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:33449:x1876219350541824:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302896.554406:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33449:x1876219350541824:12345-192.168.203.11@tcp:35:cp.0 Request processed in 778us (1035us total) trans 4294968339 rc 0/0 00000100:00100000:0.0:1789302896.554416:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 326 00000100:00100000:2.0:1789302896.557307:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371180160:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302896.557346:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302896.557445:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350541568:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 33352us (35685us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302896.557458:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 445 00000100:00100000:1.0:1789302896.701831:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.702240:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350541952 00000100:00100000:0.0:1789302896.702256:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350541952 00000100:00100000:0.0:1789302896.702343:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1522 00000100:00100000:0.0:1789302896.702353:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:33453:x1876219350541952:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302896.702379:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.702416:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd04 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0feb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702532:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702549:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702567:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702584:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702591:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702598:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702705:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184fd19 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702713:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184fd19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.702720:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184fd19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302896.702759:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2e:0x0]: returning size 158 00000004:00200000:0.0:1789302896.702764:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302896.702782:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184fd19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705307:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184fd19 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705340:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705352:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705359:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705366:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.705372:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705379:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705410:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.705413:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.705420:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705425:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd0b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705448:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705454:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705461:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705466:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.705470:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705476:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705483:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f09ae1f00) 00010000:00010000:0.0:1789302896.705485:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.705488:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705493:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd12 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705616:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd04 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0feb expref: 110 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705634:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184fd19 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0feb expref: 109 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.705642:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302896.705644:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302896.705647:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ae00, rc 0) 00000100:00100000:0.0:1789302896.705749:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:33453:x1876219350541952:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3395us (3935us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302896.705759:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1522 00000100:00100000:1.0:1789302896.713346:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.713480:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350542080 00000100:00100000:2.0:1789302896.713490:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350542080 00000100:00100000:2.0:1789302896.713580:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 425 00000100:00100000:2.0:1789302896.713590:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350542080:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302896.713606:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.713634:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000038f75c10/0x64fe43095184fd20 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ff2 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302896.713692:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371180288:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302896.713769:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b0c0 going to sleep for 11 seconds 00000100:00080000:0.0:1789302896.715924:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00010000:2.0:1789302896.727184:0:8571:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302896.727193:0:8571:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb mtime: 0 -> 1789302896 00002000:00010000:2.0:1789302896.727196:0:8571:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb atime: 0 -> 1789302896 00002000:00010000:2.0:1789302896.727198:0:8571:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x1e:0x0] updating lvb ctime: 0 -> 1789302896 00000100:00100000:2.0:1789302896.727222:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371180288:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302896.727261:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000038f75c10/0x64fe43095184fd20 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff2 expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.727284:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000038f75c10/0x64fe43095184fd20 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff2 expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.727297:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f800, rc 0) 00000100:00100000:2.0:1789302896.727354:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350542080:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 13766us (14027us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302896.727363:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 425 00000100:00100000:1.0:1789302896.746125:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.746330:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350542208 00000100:00100000:0.0:1789302896.746343:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350542208 00000100:00100000:0.0:1789302896.746420:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1523 00000100:00100000:0.0:1789302896.746433:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:33453:x1876219350542208:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302896.746459:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.746484:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd27 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a0ff9 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.746608:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.746626:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.746640:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302896.746682:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:0.0:1789302896.746686:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302896.746706:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.746735:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd27 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 111 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.746756:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 110 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.746766:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.746769:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302896.746774:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00000100:00100000:0.0:1789302896.746857:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33453:x1876219350542208:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 426us (749us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302896.746868:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1523 00000100:00100000:1.0:1789302896.755470:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.755876:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350542336 00000100:00100000:3.0:1789302896.755890:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350542336 00000100:00100000:3.0:1789302896.756085:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 426 00000100:00100000:3.0:1789302896.756098:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350542336:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302896.756122:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302896.756148:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184fd35 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1000 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302896.756229:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371180416:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302896.756337:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61000 going to sleep for 11 seconds 00000100:00100000:2.0:1789302896.767451:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371180416:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302896.767715:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184fd35 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1000 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.767743:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d13f855e/0x64fe43095184fd35 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1000 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.767757:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:2.0:1789302896.767939:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350542336:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 11843us (12481us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302896.767949:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 426 00000040:00080000:0.0:1789302896.780756:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302896.780763:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302896.781267:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 3 00000040:00080000:0.0:1789302896.781277:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302896.788988:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.789193:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350542592 00000100:00100000:3.0:1789302896.789203:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350542592 00000100:00100000:3.0:1789302896.789270:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1524 00000100:00100000:3.0:1789302896.789281:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33453:x1876219350542592:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302896.789769:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].45, 136 off14312 00000100:00100000:3.0:1789302896.789958:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33453:x1876219350542592:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 678us (978us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302896.789966:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1524 00000100:00100000:1.0:1789302896.806154:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.807069:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350542720 00000100:00100000:0.0:1789302896.807081:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350542720 00000100:00100000:0.0:1789302896.807135:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1525 00000100:00100000:0.0:1789302896.807145:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33454:x1876219350542720:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302896.807164:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.807184:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd3c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1007 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807248:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807272:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807283:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807299:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807307:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807314:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807376:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807393:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807400:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807406:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.807410:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807418:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807429:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.807433:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.807437:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807443:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd43 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807468:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807475:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807480:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807486:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.807489:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807495:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807529:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f09ae1f00) 00010000:00010000:0.0:1789302896.807532:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.807536:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807542:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd4a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807576:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd3c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1007 expref: 111 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807585:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.807588:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.807591:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd3c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1007 expref: 111 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.807600:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302896.812442:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33454:x1876219350542720:12345-192.168.203.11@tcp:101:bash.0 Request processed in 5288us (6287us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302896.812581:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1525 00000100:00100000:1.0:1789302896.834136:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.834242:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350542848 00000100:00100000:2.0:1789302896.834252:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350542848 00000100:00100000:2.0:1789302896.834315:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1526 00000100:00100000:2.0:1789302896.834338:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33455:x1876219350542848:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302896.834362:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.834394:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fd51 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a100e expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834465:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834480:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834489:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834886:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834896:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834904:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834956:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834964:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834974:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834980:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.834984:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.834992:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835002:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.835005:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.835010:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835015:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fd58 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835027:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835033:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835039:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835044:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00031c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.835046:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835055:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835062:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f124a9b00) 00010000:00010000:2.0:1789302896.835064:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.835066:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835071:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00031c00/0x64fe43095184fd5f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835105:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fd51 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a100e expref: 55 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835115:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.835118:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.835122:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fd51 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a100e expref: 55 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.835132:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f800, rc 0) 00000100:00100000:2.0:1789302896.835198:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33455:x1876219350542848:12345-192.168.203.11@tcp:101:rm.0 Request processed in 874us (1071us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302896.835205:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1526 00000100:00100000:1.0:1789302896.844821:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.847153:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350542976 00000100:00100000:2.0:1789302896.847165:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350542976 00000100:00100000:2.0:1789302896.847288:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1527 00000100:00100000:2.0:1789302896.847299:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33455:x1876219350542976:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302896.847322:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.847352:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fd66 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1015 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847434:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847449:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847468:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847483:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847490:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847497:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847562:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847570:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847577:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847583:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.847588:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847596:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847607:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.847621:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.847627:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847634:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fd6d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847647:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847654:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847661:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847666:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302896.847669:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847675:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847682:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f124a9500) 00010000:00010000:2.0:1789302896.847685:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.847688:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847694:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fd74 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847726:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fd66 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1015 expref: 55 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847736:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302896.847739:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302896.847743:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fd66 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1015 expref: 55 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.847752:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030200, rc 0) 00000100:00100000:2.0:1789302896.847820:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33455:x1876219350542976:12345-192.168.203.11@tcp:101:rm.0 Request processed in 523us (3008us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302896.847828:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1527 00000100:00100000:1.0:1789302896.869623:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.870143:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350543104 00000100:00100000:0.0:1789302896.870154:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350543104 00000100:00100000:0.0:1789302896.870213:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1528 00000100:00100000:0.0:1789302896.870223:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33456:x1876219350543104:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302896.870242:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.870273:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd7b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a101c expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870480:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870496:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870545:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870564:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870572:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870578:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870742:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870750:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870757:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870763:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.870767:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870774:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870785:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.870788:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.870793:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870798:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fd82 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870814:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870820:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870826:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870831:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.870833:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870839:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870850:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f09ae1f00) 00010000:00010000:0.0:1789302896.870852:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.870855:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870860:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fd89 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870894:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd7b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a101c expref: 55 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870904:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302896.870906:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.870909:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fd7b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a101c expref: 55 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.870917:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302896.870968:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33456:x1876219350543104:12345-192.168.203.11@tcp:101:cp.0 Request processed in 747us (1364us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302896.870974:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1528 00000100:00100000:1.0:1789302896.874749:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.874891:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350543232 00000100:00100000:3.0:1789302896.874901:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350543232 00000100:00100000:3.0:1789302896.874958:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1529 00000100:00100000:3.0:1789302896.874967:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33456:x1876219350543232:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302896.874986:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302896.875008:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fd90 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1023 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875079:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875093:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875103:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875116:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875127:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875132:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875186:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875193:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875200:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875206:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302896.875210:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875218:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875230:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302896.875233:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.875238:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875244:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fd97 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875272:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875279:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875285:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875289:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302896.875292:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875299:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875306:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302896.875309:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.875311:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875317:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fd9e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875358:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fd90 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1023 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875368:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302896.875370:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.875373:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fd90 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1023 expref: 55 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.875397:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:3.0:1789302896.875888:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33456:x1876219350543232:12345-192.168.203.11@tcp:101:cp.0 Request processed in 920us (1148us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302896.875896:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1529 00000100:00100000:1.0:1789302896.882649:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.882746:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350543488 00000100:00100000:2.0:1789302896.882756:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350543488 00000100:00100000:2.0:1789302896.882819:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1530 00000100:00100000:2.0:1789302896.882830:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:33456:x1876219350543488:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302896.882849:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.882869:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fda5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a102a expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.882985:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.883005:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 109 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.883015:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.883060:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 109 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.883075:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 109 pid: 6919 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302896.883087:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371180544:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302896.883182:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302896.894850:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302896.894945:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350543616 00000100:00100000:3.0:1789302896.894955:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350543616 00010000:00010000:3.0:1789302896.895007:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 110 pid: 6919 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302896.895057:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:3.0:1789302896.895068:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd06c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2347:x1876219350543616:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302896.895082:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302896.895088:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 110 pid: 6919 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.895101:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 110 pid: 6919 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.895121:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302896.895125:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.895135:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.895144:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.895163:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302896.895216:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd06c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2347:x1876219350543616:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 151us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302896.895223:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 446 00010000:00010000:2.0:1789302896.897891:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095184fd2e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0ff9 expref: 109 pid: 6919 timeout: 833 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302896.897916:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371180544:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302896.897941:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.897951:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.897957:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.897983:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.897996:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.898006:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302896.898972:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x1f:0x0], index 0: 0 00010000:00010000:3.0:1789302896.906458:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906481:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906494:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906559:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906567:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906573:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906579:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302896.906583:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906591:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906602:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302896.906605:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302896.906623:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906630:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fdac lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302896.906759:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000707d3848 reply state 000000004c6d71e5 transno 4294968340 ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302896.906770:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302896.906810:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fda5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 57 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906826:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 56 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302896.906833:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302896.906836:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302896.906839:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1200, rc 0) 00000100:00100000:3.0:1789302896.906915:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33456:x1876219350543488:12345-192.168.203.11@tcp:101:cp.0 Request processed in 24087us (24275us total) trans 4294968340 rc 0/0 00000100:00100000:3.0:1789302896.906924:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1530 00010000:00010000:0.0:1789302896.911146:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.911164:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.911173:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.911185:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302896.911190:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.911199:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.911216:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f124a9500) 00010000:00010000:0.0:1789302896.911219:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302896.911224:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.911231:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fdb3 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302896.924288:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302896.924324:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302896.924363:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.924477:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350543744 00000100:00100000:0.0:1789302896.924486:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350543744 00000100:00100000:0.0:1789302896.924574:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350544000 00000100:00100000:0.0:1789302896.924578:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350544000 00000100:00100000:0.0:1789302896.924600:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 427 00000100:00100000:0.0:1789302896.924618:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350543744:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302896.924635:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302896.924656:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1031 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302896.924683:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302896.924701:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=733s v=(0 0) 00010000:00010000:0.0:1789302896.924706:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302896.924716:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.924734:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 10 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.924746:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302896.924794:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350543744:12345-192.168.203.11@tcp:101:cp.0 Request processed in 186us (514us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302896.924801:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 427 00000100:00100000:0.0:1789302896.924819:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 428 00000100:00100000:0.0:1789302896.924827:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350544000:12345-192.168.203.11@tcp:16:kworker.0 00000100:00100000:0.0:1789302896.924879:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350544000:12345-192.168.203.11@tcp:16:kworker.0 Request processed in 53us (515us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302896.924886:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 428 00000100:00100000:2.0:1789302896.925706:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350543872 00000100:00100000:2.0:1789302896.925712:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350543872 00002000:00100000:2.0:1789302896.925745:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1e:0x0] (0->157) req@ffff960e0267f100 x1876219350543872/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:472/0 lens 648/0 e 0 to 0 dl 1789302907 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302896.925755:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0267f100 with x1876219350543872 ext(0->157) 00002000:00010000:2.0:1789302896.925761:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0267f100 00000100:00100000:2.0:1789302896.925787:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 128 00000100:00100000:2.0:1789302896.925795:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350543872:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:2.0:1789302896.926233:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350543872:12345-192.168.203.11@tcp:4:cp.0 Request processed in 438us (1906us total) trans 4294967382 rc 0/0 00000100:00100000:2.0:1789302896.926242:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 128 00002000:00100000:2.0:1789302896.926249:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x1e:0x0] (0->157) req@ffff960e0267f100 x1876219350543872/t4294967382(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:472/0 lens 648/448 e 0 to 0 dl 1789302907 ref 1 fl Complete:/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302896.926258:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0267f100 with x1876219350543872 ext(0->157) 00002000:00010000:2.0:1789302896.926262:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0267f100 00000100:00080000:1.0:1789302896.926582:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302896.930452:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302896.930716:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350544128 00000100:00100000:2.0:1789302896.930726:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350544128 00000100:00100000:2.0:1789302896.930787:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 429 00000100:00100000:2.0:1789302896.930799:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:33456:x1876219350544128:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302896.930817:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302896.930835:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 2/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1038 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.930867:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.930877:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302896.930881:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302896.930887:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.930918:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.930933:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 11 pid: 13439 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302896.930947:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371180672:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302896.931048:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302896.934074:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371180672:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302896.934108:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 3/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302896.934134:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031c00, rc 0) 00000100:00100000:2.0:1789302896.934228:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:33456:x1876219350544128:12345-192.168.203.11@tcp:101:cp.0 Request processed in 3431us (3782us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302896.934237:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 429 00000100:00100000:3.0:1789302896.937444:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.937935:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350544256 00000100:00100000:0.0:1789302896.937945:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350544256 00010000:00010000:0.0:1789302896.937983:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 11 pid: 13439 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302896.938022:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00100000:0.0:1789302896.938031:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350544256:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302896.938878:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302896.938904:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 11 pid: 13439 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.938924:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 11 pid: 13439 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.938947:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdc8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1031 expref: 11 pid: 13439 timeout: 833 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.938966:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302896.938971:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302896.938979:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.938988:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302896.939026:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 10 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302896.939052:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371180800:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302896.939120:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea0c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302896.947883:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371180800:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302896.947907:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302896.947959:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350544256:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 9929us (10522us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302896.947969:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00100000:1.0:1789302896.954408:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302896.954543:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350544512 00000100:00100000:0.0:1789302896.954553:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350544512 00000100:00100000:0.0:1789302896.954621:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 327 00000100:00100000:0.0:1789302896.954631:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:33456:x1876219350544512:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302896.954967:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33456:x1876219350544512:12345-192.168.203.11@tcp:35:cp.0 Request processed in 337us (567us total) trans 4294968341 rc 0/0 00000100:00100000:0.0:1789302896.954975:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 327 00000100:00100000:1.0:1789302897.031617:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.031931:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350544640 00000100:00100000:0.0:1789302897.031941:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350544640 00000100:00100000:0.0:1789302897.031999:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1531 00000100:00100000:0.0:1789302897.032009:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+109:33459:x1876219350544640:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.032036:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.032055:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fdd6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a103f expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032119:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032133:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032145:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032159:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032167:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032177:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032253:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fdeb lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032261:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fdeb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032269:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fdeb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302897.032299:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2f:0x0]: returning size 158 00000004:00200000:0.0:1789302897.032303:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302897.032317:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fdeb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032329:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fdeb lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032337:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032344:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032351:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032357:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.032361:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032368:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032391:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.032395:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.032399:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032406:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fddd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032420:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032426:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032444:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032449:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.032452:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032459:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032466:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f09ae1000) 00010000:00010000:0.0:1789302897.032469:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.032472:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032482:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fde4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032697:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe43095184fdd6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a103f expref: 111 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032713:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fdeb lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a103f expref: 110 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.032720:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:0.0:1789302897.032722:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302897.032725:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a800, rc 0) 00000100:00100000:0.0:1789302897.032779:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33459:x1876219350544640:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 771us (1171us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.032788:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1531 00000100:00100000:3.0:1789302897.041717:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.041776:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350544768 00000100:00100000:0.0:1789302897.041788:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350544768 00000100:00100000:0.0:1789302897.041861:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 430 00000100:00100000:0.0:1789302897.041870:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350544768:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.041886:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.041902:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdf2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1046 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.041956:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371180928:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302897.042050:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea0c0 going to sleep for 11 seconds 00000100:00080000:1.0:1789302897.042175:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00002000:00010000:0.0:1789302897.051316:0:13439:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1789302897.051325:0:13439:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb mtime: 0 -> 1789302896 00002000:00010000:0.0:1789302897.051328:0:13439:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb atime: 0 -> 1789302896 00002000:00010000:0.0:1789302897.051331:0:13439:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x1f:0x0] updating lvb ctime: 0 -> 1789302896 00000100:00100000:0.0:1789302897.051354:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b0a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371180928:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302897.051399:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdf2 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1046 expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.051421:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003e2e9d08/0x64fe43095184fdf2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1046 expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.051613:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302897.051670:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350544768:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 9800us (9960us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.051679:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 430 00000100:00100000:2.0:1789302897.068134:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.068293:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350544896 00000100:00100000:3.0:1789302897.068302:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350544896 00000100:00100000:3.0:1789302897.068358:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1532 00000100:00100000:3.0:1789302897.068367:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33459:x1876219350544896:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302897.068398:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.068417:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fdf9 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a104d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.068492:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.068668:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.068682:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302897.068714:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:3.0:1789302897.068717:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302897.068732:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.068756:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fdf9 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 112 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.068776:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 111 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.068785:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.068788:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302897.068792:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302897.068850:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33459:x1876219350544896:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 484us (724us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.068861:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1532 00000100:00100000:3.0:1789302897.077617:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.077720:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350545024 00000100:00100000:2.0:1789302897.077730:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350545024 00000100:00100000:2.0:1789302897.077790:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 431 00000100:00100000:2.0:1789302897.077800:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2345:x1876219350545024:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302897.077817:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.077833:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001b42e3e4/0x64fe43095184fe07 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1054 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.077886:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371181056:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302897.077972:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302897.090322:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371181056:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302897.090358:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000001b42e3e4/0x64fe43095184fe07 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1054 expref: 17 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.090394:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001b42e3e4/0x64fe43095184fe07 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1054 expref: 17 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.090410:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030800, rc 0) 00000100:00100000:2.0:1789302897.090476:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2345:x1876219350545024:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 12677us (12867us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.090485:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 431 00000100:00100000:1.0:1789302897.098693:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.098790:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350545280 00000100:00100000:3.0:1789302897.098801:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350545280 00000100:00100000:3.0:1789302897.098866:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1533 00000100:00100000:3.0:1789302897.098875:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33459:x1876219350545280:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302897.099231:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].46, 136 off14448 00000100:00100000:3.0:1789302897.099348:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33459:x1876219350545280:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 474us (663us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.099356:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1533 00000100:00100000:3.0:1789302897.127938:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.128052:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350545408 00000100:00100000:2.0:1789302897.128062:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350545408 00000100:00100000:2.0:1789302897.128124:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1534 00000100:00100000:2.0:1789302897.128134:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33460:x1876219350545408:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302897.128153:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.128171:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe0e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a105b expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128234:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128247:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128257:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128271:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128279:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128285:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128333:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128340:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128348:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128354:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.128359:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128366:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128378:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.128392:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.128396:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128403:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe15 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128416:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128423:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128429:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128433:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.128437:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128443:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128449:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f124a9b00) 00010000:00010000:2.0:1789302897.128461:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.128463:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128469:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe1c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128499:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe0e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a105b expref: 112 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128707:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.128710:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.128714:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe0e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a105b expref: 112 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.128726:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030800, rc 0) 00000100:00100000:2.0:1789302897.128793:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33460:x1876219350545408:12345-192.168.203.11@tcp:101:bash.0 Request processed in 659us (863us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.128799:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1534 00000100:00100000:1.0:1789302897.161348:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.163009:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350545536 00000100:00100000:0.0:1789302897.163021:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350545536 00000100:00100000:0.0:1789302897.163086:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1535 00000100:00100000:0.0:1789302897.163095:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33461:x1876219350545536:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302897.163119:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.163138:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fe23 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1062 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163198:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163212:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163228:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163243:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163250:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163256:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163302:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163310:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163317:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163322:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.163327:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163334:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163345:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.163348:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.163353:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163359:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe2a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163392:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163399:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163405:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163410:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.163413:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163418:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163425:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f09ae1000) 00010000:00010000:0.0:1789302897.163428:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.163430:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.163436:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe31 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.164062:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fe23 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1062 expref: 58 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.164080:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.164083:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.164086:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095184fe23 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1062 expref: 58 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.164097:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345aa00, rc 0) 00000100:00100000:0.0:1789302897.164208:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33461:x1876219350545536:12345-192.168.203.11@tcp:101:rm.0 Request processed in 1113us (2868us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.164216:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1535 00000100:00100000:0.0:1789302897.174842:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.175293:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350545664 00000100:00100000:3.0:1789302897.175302:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350545664 00000100:00100000:3.0:1789302897.175356:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1536 00000100:00100000:3.0:1789302897.175366:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33461:x1876219350545664:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302897.175395:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.175416:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fe38 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1069 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175491:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175623:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175636:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175657:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175665:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175671:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175720:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175726:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175732:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175739:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.175743:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175751:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175762:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.175765:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.175769:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175776:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fe3f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175790:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175796:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175802:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175807:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.175809:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175815:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175823:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f0cdf2c00) 00010000:00010000:3.0:1789302897.175825:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.175827:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175833:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fe46 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175863:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fe38 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1069 expref: 58 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175874:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.175876:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.175880:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe43095184fe38 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1069 expref: 58 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.175888:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00000100:00100000:3.0:1789302897.175934:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33461:x1876219350545664:12345-192.168.203.11@tcp:101:rm.0 Request processed in 570us (1101us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302897.175942:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1536 00000100:00100000:3.0:1789302897.210438:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.210670:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350545792 00000100:00100000:2.0:1789302897.210680:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350545792 00000100:00100000:2.0:1789302897.210735:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1537 00000100:00100000:2.0:1789302897.210745:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33462:x1876219350545792:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302897.210765:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.210785:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe4d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1070 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210845:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210859:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210870:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210884:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210891:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210898:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210964:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210971:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210978:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210985:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.210989:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.210997:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211009:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.211011:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.211017:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211024:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe54 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211037:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211043:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211049:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211055:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.211057:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211062:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211069:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f124a9b00) 00010000:00010000:2.0:1789302897.211071:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.211074:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211080:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe5b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211109:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe4d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1070 expref: 58 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211118:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.211121:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.211124:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe4d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1070 expref: 58 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.211134:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030800, rc 0) 00000100:00100000:2.0:1789302897.211188:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33462:x1876219350545792:12345-192.168.203.11@tcp:101:cp.0 Request processed in 444us (766us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.211195:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1537 00000100:00100000:1.0:1789302897.216184:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.216410:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350545920 00000100:00100000:0.0:1789302897.216420:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350545920 00000100:00100000:0.0:1789302897.216486:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1538 00000100:00100000:0.0:1789302897.216495:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33462:x1876219350545920:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302897.216651:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.216672:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe62 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1077 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216732:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216747:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216757:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216776:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216783:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216789:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216837:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216844:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216851:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216857:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.216861:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216868:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216880:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.216883:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.216887:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216893:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184fe69 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216906:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216912:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216918:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216922:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.216925:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216931:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216937:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f09ae1000) 00010000:00010000:0.0:1789302897.216940:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.216943:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.216948:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe43095184fe70 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.217138:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe62 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1077 expref: 58 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.217149:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.217152:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.217156:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095184fe62 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1077 expref: 58 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.217166:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00000100:00100000:0.0:1789302897.217223:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33462:x1876219350545920:12345-192.168.203.11@tcp:101:cp.0 Request processed in 729us (1048us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.217231:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1538 00000100:00100000:0.0:1789302897.224333:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.224428:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350546176 00000100:00100000:2.0:1789302897.224456:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350546176 00000100:00100000:2.0:1789302897.224804:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1539 00000100:00100000:2.0:1789302897.224817:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33462:x1876219350546176:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302897.224836:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.224855:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe77 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a107e expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.224969:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.224989:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 110 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.224998:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.225029:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 110 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.225042:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 110 pid: 5845 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.225055:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371181184:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302897.225127:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302897.227894:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.228025:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371181184:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302897.228062:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.229712:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350546304 00000100:00100000:1.0:1789302897.229724:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350546304 00010000:00010000:1.0:1789302897.229771:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 111 pid: 5845 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.229901:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 448 00000100:00100000:1.0:1789302897.229912:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:2347:x1876219350546304:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.229929:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.229936:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 111 pid: 5845 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.229948:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 111 pid: 5845 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.229969:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184fe00 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a104d expref: 111 pid: 5845 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.229988:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302897.229993:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.230003:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.230010:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.230051:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302897.230114:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2347:x1876219350546304:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 203us (2227us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.230122:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 448 00010000:00010000:2.0:1789302897.231247:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.231269:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.231296:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.231307:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.231316:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302897.232046:0:5843:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x1f:0x0], index 1: 0 00010000:00010000:2.0:1789302897.232211:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232222:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232232:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232276:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232284:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232292:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232298:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.232303:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232311:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232323:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.232326:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.232330:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232336:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe43095184fe7e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302897.232349:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000e7891422 reply state 0000000050e59848 transno 4294968342 ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.232358:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302897.232400:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095184fe77 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 60 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232415:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 59 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.232423:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:2.0:1789302897.232426:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302897.232429:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7f800, rc 0) 00000100:00100000:2.0:1789302897.232541:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33462:x1876219350546176:12345-192.168.203.11@tcp:101:cp.0 Request processed in 7726us (8215us total) trans 4294968342 rc 0/0 00000100:00100000:2.0:1789302897.232549:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1539 00000100:00100000:2.0:1789302897.237224:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:3.0:1789302897.237866:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.237879:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.237887:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.237893:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.237897:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.237905:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.237921:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f124a9b00) 00010000:00010000:3.0:1789302897.237925:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.237930:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.237936:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095184fe85 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.242757:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350546432 00000100:00100000:0.0:1789302897.242772:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350546432 00000100:00100000:0.0:1789302897.242874:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 432 00000100:00100000:0.0:1789302897.242886:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350546432:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302897.242903:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.242926:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1085 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302897.242953:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1f:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302897.242975:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789302897.242980:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302897.242989:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 13 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.243020:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 13 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.243263:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ac00, rc 0) 00000100:00100000:0.0:1789302897.243363:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350546432:12345-192.168.203.11@tcp:101:cp.0 Request processed in 478us (6147us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.243372:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 432 00000100:00100000:0.0:1789302897.250726:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.250803:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350546560 00000100:00100000:3.0:1789302897.250813:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350546560 00000100:00100000:3.0:1789302897.250866:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 433 00000100:00100000:3.0:1789302897.250875:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:33462:x1876219350546560:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302897.250892:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.250911:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a108c expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.250941:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 14 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.250950:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302897.250964:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302897.250970:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 14 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.251001:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 14 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.251014:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 14 pid: 6695 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.251028:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371181312:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302897.251103:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61600 going to sleep for 11 seconds 00000100:00100000:1.0:1789302897.260683:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.260785:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350546688 00000100:00100000:2.0:1789302897.260794:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350546688 00010000:00010000:2.0:1789302897.260834:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 15 pid: 6695 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.260854:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa4a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371181312:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302897.260887:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 3/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 15 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.260888:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 449 00000100:00100000:2.0:1789302897.260902:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2347:x1876219350546688:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302897.260910:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1800, rc 0) 00010000:00010000:2.0:1789302897.260918:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302897.260941:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 15 pid: 6695 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.260966:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:33462:x1876219350546560:12345-192.168.203.11@tcp:101:cp.0 Request processed in 10091us (10247us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.260974:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 433 00010000:00010000:2.0:1789302897.261063:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 14 pid: 6695 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.261086:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000062cdd7cc/0x64fe43095184fe9a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1085 expref: 14 pid: 6695 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.261100:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302897.261104:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302897.261111:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.261122:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.261154:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.261166:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371181440:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302897.261221:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302897.264471:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371181440:192.168.203.11@tcp:105: 00010000:00010000:0.0:1789302897.264497:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302897.264734:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350546688:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 3833us (4058us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.264744:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 449 00000100:00100000:0.0:1789302897.267306:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.276887:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350546944 00000100:00100000:3.0:1789302897.276904:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350546944 00000100:00100000:2.0:1789302897.279049:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 328 00000100:00100000:2.0:1789302897.279061:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:33462:x1876219350546944:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302897.279462:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33462:x1876219350546944:12345-192.168.203.11@tcp:35:cp.0 Request processed in 401us (12162us total) trans 4294968343 rc 0/0 00000100:00100000:2.0:1789302897.279470:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 328 00010000:00010000:1.0:1789302897.292724:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302897.389768:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.390495:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350547072 00000100:00100000:0.0:1789302897.390610:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350547072 00000100:00100000:0.0:1789302897.390678:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1540 00000100:00100000:0.0:1789302897.390692:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33465:x1876219350547072:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.390711:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.390731:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fea8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1093 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390794:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390808:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390819:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390833:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390845:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390852:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390936:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184febd lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390944:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184febd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.390951:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184febd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302897.390987:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x30:0x0]: returning size 158 00000004:00200000:0.0:1789302897.390991:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302897.391005:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184febd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391016:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184febd lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391024:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391030:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391037:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391047:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.391052:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391059:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391070:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.391073:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.391077:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391083:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184feaf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391097:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391104:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391110:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391115:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.391117:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391123:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391133:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f0cf83100) 00010000:00010000:0.0:1789302897.391136:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.391138:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391144:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184feb6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391169:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fea8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1093 expref: 112 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391182:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184febd lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1093 expref: 111 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.391189:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:0.0:1789302897.391192:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302897.391194:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ac00, rc 0) 00000100:00100000:0.0:1789302897.391248:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33465:x1876219350547072:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 562us (1491us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.391255:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1540 00000100:00100000:2.0:1789302897.429092:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.429140:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.429405:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350547328 00000100:00100000:3.0:1789302897.429415:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350547328 00000100:00100000:3.0:1789302897.429483:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 75 00000100:00100000:3.0:1789302897.429492:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:2345:x1876219350547328:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:3.0:1789302897.429578:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.429598:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fec4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a10a1 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429651:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429665:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429676:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429690:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429701:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429707:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429757:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429765:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429771:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429778:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.429782:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429791:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429802:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.429806:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.429811:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429818:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184fecb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429831:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429838:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429844:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429848:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.429851:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429857:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429862:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cdf2c00) 00010000:00010000:3.0:1789302897.429864:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.429867:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429873:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184fed2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429898:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fec4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10a1 expref: 112 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429908:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.429910:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302897.429914:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fec4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10a1 expref: 112 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.429928:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc -2) 00000100:00100000:3.0:1789302897.429994:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:2345:x1876219350547328:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 503us (852us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302897.430001:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 75 00000100:00100000:0.0:1789302897.431627:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350547200 00000100:00100000:0.0:1789302897.431645:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350547200 00000100:00100000:0.0:1789302897.431697:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 434 00000100:00100000:0.0:1789302897.431705:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350547200:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.431718:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.431732:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184fed9 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a109a expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.431790:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371181568:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302897.431866:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098ea0c0 going to sleep for 11 seconds 00002000:00010000:0.0:1789302897.445168:0:8571:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb size: 0 -> 158 00002000:00010000:0.0:1789302897.445176:0:8571:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb mtime: 0 -> 1789302896 00002000:00010000:0.0:1789302897.445179:0:8571:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb atime: 0 -> 1789302896 00002000:00010000:0.0:1789302897.445181:0:8571:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x1f:0x0] updating lvb ctime: 0 -> 1789302896 00000100:00100000:0.0:1789302897.445203:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371181568:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302897.445236:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184fed9 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a109a expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.445259:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000077698a76/0x64fe43095184fed9 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a109a expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.445273:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345be00, rc 0) 00000100:00100000:0.0:1789302897.445334:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350547200:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 13630us (16250us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.445342:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 434 00000100:00100000:3.0:1789302897.450773:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.450850:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350547456 00000100:00100000:0.0:1789302897.450860:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350547456 00000100:00100000:0.0:1789302897.450911:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1541 00000100:00100000:0.0:1789302897.450921:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33465:x1876219350547456:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.450938:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.450956:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fee0 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a10d9 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.451020:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.451043:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.451055:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302897.451089:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:0.0:1789302897.451093:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302897.451107:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.451130:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fee0 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 113 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.451149:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 112 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.451158:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.451161:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302897.451164:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b400, rc 0) 00000100:00100000:0.0:1789302897.451212:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33465:x1876219350547456:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 293us (448us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.451219:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1541 00000100:00100000:2.0:1789302897.461521:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.461599:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350547584 00000100:00100000:3.0:1789302897.461608:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350547584 00000100:00100000:3.0:1789302897.461667:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 435 00000100:00100000:3.0:1789302897.461677:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350547584:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302897.461694:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.461712:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184feee lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a10e0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.461764:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371181696:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302897.461838:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61000 going to sleep for 11 seconds 00000100:00100000:3.0:1789302897.464464:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa7100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371181696:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302897.464498:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184feee lrc: 1/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10e0 expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.464552:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c108c604/0x64fe43095184feee lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10e0 expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.464565:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302897.464615:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350547584:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2940us (3122us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302897.464624:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 435 00000100:00100000:2.0:1789302897.470874:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.470958:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350547840 00000100:00100000:0.0:1789302897.470968:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350547840 00000100:00100000:0.0:1789302897.471035:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1542 00000100:00100000:0.0:1789302897.471045:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33465:x1876219350547840:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:0.0:1789302897.471267:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].47, 136 off14584 00000100:00100000:0.0:1789302897.471344:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33465:x1876219350547840:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 301us (479us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.471352:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1542 00000100:00100000:2.0:1789302897.495862:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.495900:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.495991:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350548096 00000100:00100000:0.0:1789302897.496001:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350548096 00000100:00100000:0.0:1789302897.496061:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 76 00000100:00100000:0.0:1789302897.496070:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2345:x1876219350548096:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302897.496107:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.496125:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fef5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a10ee expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496177:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496190:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496201:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496237:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496245:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496253:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496302:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496309:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496316:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496322:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.496326:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496334:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496346:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.496348:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.496353:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496359:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184fefc lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496373:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496390:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496397:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496403:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.496406:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496411:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496418:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x996ea506 (ffff960f0cf83100) 00010000:00010000:0.0:1789302897.496420:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.496423:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496669:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff03 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496701:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fef5 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10ee expref: 113 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496711:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.496713:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302897.496717:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184fef5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10ee expref: 113 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.496726:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc -2) 00000100:00100000:0.0:1789302897.496792:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2345:x1876219350548096:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 723us (890us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302897.496800:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 76 00000100:00100000:3.0:1789302897.498160:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350547968 00000100:00100000:3.0:1789302897.498170:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350547968 00000100:00100000:3.0:1789302897.498218:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1543 00000100:00100000:3.0:1789302897.498226:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33468:x1876219350547968:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302897.498243:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.498269:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ff0a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a10e7 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498318:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498330:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498339:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498352:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498358:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498365:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498417:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498437:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498444:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498456:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.498460:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498467:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498477:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.498480:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.498484:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498490:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff11 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498546:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498553:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498559:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498565:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.498568:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498574:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498581:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cdf2c00) 00010000:00010000:3.0:1789302897.498583:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.498586:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498592:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff18 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498618:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ff0a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10e7 expref: 113 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498627:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.498629:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.498632:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ff0a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10e7 expref: 113 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.498641:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302897.498696:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33468:x1876219350547968:12345-192.168.203.11@tcp:101:bash.0 Request processed in 470us (2843us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302897.498703:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1543 00000100:00100000:2.0:1789302897.521339:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.521552:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350548224 00000100:00100000:3.0:1789302897.521561:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350548224 00000100:00100000:3.0:1789302897.521616:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1544 00000100:00100000:3.0:1789302897.521625:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33469:x1876219350548224:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302897.521657:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.521678:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ff1f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a10f5 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521741:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521755:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521766:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521779:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521785:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521792:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521843:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521849:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521856:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521862:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.521867:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521875:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521886:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.521889:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.521893:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521899:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff26 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521913:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521919:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521925:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521930:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.521933:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521938:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521944:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cdf2c00) 00010000:00010000:3.0:1789302897.521946:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.521949:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521955:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff2d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521983:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ff1f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10f5 expref: 61 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.521992:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.521995:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.521999:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ff1f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10f5 expref: 61 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.522007:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302897.522061:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33469:x1876219350548224:12345-192.168.203.11@tcp:101:rm.0 Request processed in 438us (730us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302897.522069:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1544 00000100:00100000:2.0:1789302897.530653:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.530745:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350548352 00000100:00100000:0.0:1789302897.530755:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350548352 00000100:00100000:0.0:1789302897.530808:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1545 00000100:00100000:0.0:1789302897.530817:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33469:x1876219350548352:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302897.530836:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.530855:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff34 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a10fc expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.530915:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.530928:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.530939:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.530952:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.530959:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.530966:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531015:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531023:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531029:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531047:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.531051:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531059:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531069:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.531072:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.531077:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531084:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff3b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531098:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531104:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531110:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531115:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.531117:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531122:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531133:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cf83100) 00010000:00010000:0.0:1789302897.531136:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.531138:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531145:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff42 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531174:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff34 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10fc expref: 61 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531183:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.531186:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.531189:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff34 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10fc expref: 61 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.531198:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc 0) 00000100:00100000:0.0:1789302897.531251:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33469:x1876219350548352:12345-192.168.203.11@tcp:101:rm.0 Request processed in 436us (617us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.531259:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1545 00000100:00100000:2.0:1789302897.564876:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.565949:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350548480 00000100:00100000:0.0:1789302897.565958:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350548480 00000100:00100000:0.0:1789302897.566017:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1546 00000100:00100000:0.0:1789302897.566026:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33470:x1876219350548480:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302897.566043:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.566062:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff49 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1103 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566125:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566139:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566151:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566165:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566173:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566179:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566225:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566232:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566239:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566245:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.566249:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566274:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566286:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.566289:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.566293:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566300:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff50 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566314:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566321:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566327:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566332:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.566335:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566340:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566346:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cf83100) 00010000:00010000:0.0:1789302897.566349:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.566351:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566357:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff57 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566396:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff49 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1103 expref: 61 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566405:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.566408:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.566411:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff49 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1103 expref: 61 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.566420:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc 0) 00000100:00100000:0.0:1789302897.566586:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33470:x1876219350548480:12345-192.168.203.11@tcp:101:cp.0 Request processed in 561us (1719us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.566593:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1546 00000100:00100000:2.0:1789302897.576583:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.576656:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350548608 00000100:00100000:0.0:1789302897.576666:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350548608 00000100:00100000:0.0:1789302897.576719:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1547 00000100:00100000:0.0:1789302897.576728:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33470:x1876219350548608:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302897.576746:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.576765:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff5e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a110a expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576833:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576846:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576857:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576871:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576877:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576884:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576930:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576938:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576944:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576950:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.576955:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576962:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576973:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.576976:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.576980:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576986:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095184ff65 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.576998:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577004:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577010:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577015:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.577018:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577024:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577040:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cf83100) 00010000:00010000:0.0:1789302897.577043:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.577046:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577052:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff6c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577078:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff5e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a110a expref: 61 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577094:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.577096:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.577100:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff5e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a110a expref: 61 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.577108:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc 0) 00000100:00100000:0.0:1789302897.577165:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33470:x1876219350548608:12345-192.168.203.11@tcp:101:cp.0 Request processed in 438us (591us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.577173:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1547 00000100:00100000:2.0:1789302897.581026:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.581109:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350548864 00000100:00100000:3.0:1789302897.581118:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350548864 00000100:00100000:3.0:1789302897.581169:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1548 00000100:00100000:3.0:1789302897.581177:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33470:x1876219350548864:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302897.581197:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.581216:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff73 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1111 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.581322:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.581342:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 111 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.581351:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.581392:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 111 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.581406:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 111 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.581420:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371181824:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302897.581900:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61000 going to sleep for 11 seconds 00000100:00100000:3.0:1789302897.594085:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.594182:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350548992 00000100:00100000:1.0:1789302897.594191:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350548992 00010000:00010000:1.0:1789302897.594230:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 112 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.594275:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 450 00000100:00100000:1.0:1789302897.594283:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2347:x1876219350548992:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.594297:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.594304:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 112 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.594317:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 112 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.594336:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302897.594341:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.594351:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.594358:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.594386:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302897.594442:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2347:x1876219350548992:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 159us (365us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.594449:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 450 00010000:00010000:0.0:1789302897.594977:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe43095184fee7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a10d9 expref: 111 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.594995:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa7100 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371181824:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302897.595017:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595026:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595031:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595051:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595061:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595074:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:0.0:1789302897.595653:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x20:0x0], index 0: 0 00010000:00010000:0.0:1789302897.595817:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595826:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595833:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595866:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595872:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595878:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595884:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.595888:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595895:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595904:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.595906:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.595911:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595917:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ff7a lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302897.595927:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000000ad16318 reply state 00000000d57d02f1 transno 4294968344 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.595935:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:0.0:1789302897.595964:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ff73 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 63 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595977:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 62 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.595983:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:0.0:1789302897.595985:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302897.595988:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345be00, rc 0) 00000100:00100000:0.0:1789302897.596040:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33470:x1876219350548864:12345-192.168.203.11@tcp:101:cp.0 Request processed in 14863us (15022us total) trans 4294968344 rc 0/0 00000100:00100000:0.0:1789302897.596047:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1548 00010000:00010000:0.0:1789302897.599233:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.599248:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.599256:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.599267:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.599272:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.599279:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.599294:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cf83100) 00010000:00010000:0.0:1789302897.599296:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.599301:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.599308:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095184ff81 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.603051:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.603121:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350549120 00000100:00100000:0.0:1789302897.603131:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350549120 00000100:00100000:0.0:1789302897.603182:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 436 00000100:00100000:0.0:1789302897.603191:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350549120:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302897.603209:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.603227:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1118 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302897.603253:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x20:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302897.603282:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789302897.603286:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302897.603295:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.603318:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 11 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.603331:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b400, rc 0) 00000100:00100000:0.0:1789302897.603392:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2346:x1876219350549120:12345-192.168.203.11@tcp:101:cp.0 Request processed in 202us (359us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.603399:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 436 00000100:00100000:3.0:1789302897.606099:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.606175:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350549248 00000100:00100000:2.0:1789302897.606184:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350549248 00000100:00100000:2.0:1789302897.606231:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00100000:2.0:1789302897.606239:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:33470:x1876219350549248:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302897.606254:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.606269:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 2/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a111f expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.606293:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.606309:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302897.606312:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302897.606317:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 12 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.606345:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.606359:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 12 pid: 13439 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.606372:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371181952:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302897.606593:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9ba80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302897.616327:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.617089:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371181952:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302897.617126:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 3/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 12 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.617147:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030200, rc 0) 00000100:00100000:2.0:1789302897.617197:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:33470:x1876219350549248:12345-192.168.203.11@tcp:101:cp.0 Request processed in 10958us (11105us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.617206:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 437 00000100:00100000:3.0:1789302897.617779:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350549376 00000100:00100000:3.0:1789302897.617787:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350549376 00000100:00100000:3.0:1789302897.617855:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 77 00000100:00100000:3.0:1789302897.617864:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:2345:x1876219350549376:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:3.0:1789302897.617912:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.617930:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffa4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1126 expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.617975:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.617993:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618002:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618015:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618022:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618028:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618087:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618093:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618101:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618107:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.618110:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618117:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618128:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.618131:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.618135:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618142:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffab lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618153:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618158:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618163:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618168:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.618171:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618177:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618182:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f0cdf2600) 00010000:00010000:3.0:1789302897.618184:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.618187:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618193:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffb2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618217:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffa4 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1126 expref: 63 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618226:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.618228:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302897.618232:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffa4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1126 expref: 63 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.618240:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc -2) 00000100:00100000:3.0:1789302897.618275:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:2345:x1876219350549376:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 413us (1957us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302897.618282:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 77 00000100:00100000:3.0:1789302897.618333:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.618696:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350549504 00000100:00100000:1.0:1789302897.618702:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350549504 00010000:00010000:1.0:1789302897.618724:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 12 pid: 13439 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.618751:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:1.0:1789302897.618758:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350549504:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.618769:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.618785:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 12 pid: 13439 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.618797:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 12 pid: 13439 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.618813:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000005fc3651e/0x64fe43095184ff96 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1118 expref: 12 pid: 13439 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.618825:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302897.618830:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302897.618837:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.618848:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.618877:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 11 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.618891:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371182080:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302897.618950:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f123a6480 going to sleep for 11 seconds 00000100:00100000:1.0:1789302897.624408:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb8700 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0000_UUID:13636:1876219371182080:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302897.624440:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302897.624495:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350549504:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 5737us (6160us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.624535:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:2.0:1789302897.626622:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.627102:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350549760 00000100:00100000:0.0:1789302897.627112:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350549760 00000100:00100000:0.0:1789302897.627164:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:0.0:1789302897.627173:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22332450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33470:x1876219350549760:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302897.627849:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22332450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33470:x1876219350549760:12345-192.168.203.11@tcp:35:cp.0 Request processed in 675us (1231us total) trans 4294968345 rc 0/0 00000100:00100000:0.0:1789302897.627858:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 329 00000100:00100000:0.0:1789302897.648606:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.648785:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350549888 00000100:00100000:3.0:1789302897.648796:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350549888 00000100:00100000:3.0:1789302897.648857:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 78 00000100:00100000:3.0:1789302897.648867:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:2346:x1876219350549888:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:3.0:1789302897.648922:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.648943:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffb9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a115e expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.648996:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649009:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649020:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649036:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649043:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649049:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649101:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649111:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649118:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649125:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.649129:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649136:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649146:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.649150:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.649154:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649160:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffc0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649173:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649179:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649186:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649192:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.649195:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649201:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649209:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f0cdf2600) 00010000:00010000:3.0:1789302897.649211:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.649214:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649219:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffc7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649248:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffb9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a115e expref: 64 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649258:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.649260:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302897.649264:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffb9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a115e expref: 64 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.649274:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc -2) 00000100:00100000:3.0:1789302897.649343:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:2346:x1876219350549888:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 477us (746us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302897.649351:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 78 00000100:00100000:0.0:1789302897.690630:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.691735:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350550016 00000100:00100000:3.0:1789302897.691746:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350550016 00000100:00100000:3.0:1789302897.691800:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1549 00000100:00100000:3.0:1789302897.691809:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33477:x1876219350550016:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302897.691830:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.691849:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffce lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1196 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.691908:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.691921:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.691932:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.691944:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.691952:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.691958:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692856:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ffe3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692867:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ffe3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692876:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ffe3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302897.692910:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x31:0x0]: returning size 158 00000004:00200000:3.0:1789302897.692914:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302897.692930:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ffe3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692941:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ffe3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692948:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692955:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692962:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692968:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.692971:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692978:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.692989:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.692992:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.692996:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693015:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095184ffd5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693027:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693034:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693040:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693044:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.693047:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693054:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693060:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f0cdf2600) 00010000:00010000:3.0:1789302897.693062:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.693065:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693071:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095184ffdc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693107:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184ffce lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1196 expref: 113 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693121:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ffe3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1196 expref: 112 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.693128:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:3.0:1789302897.693130:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302897.693132:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1e00, rc 0) 00000100:00100000:3.0:1789302897.693195:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33477:x1876219350550016:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 1386us (2573us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.693202:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1549 00000100:00100000:2.0:1789302897.702938:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.703024:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350550144 00000100:00100000:0.0:1789302897.703042:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350550144 00000100:00100000:0.0:1789302897.703100:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:0.0:1789302897.703109:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350550144:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.703127:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.703145:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe43095184ffea lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a119d expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.703207:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371182208:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302897.703284:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1035b840 going to sleep for 11 seconds 00002000:00010000:3.0:1789302897.706435:0:6695:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1789302897.706444:0:6695:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb mtime: 0 -> 1789302897 00002000:00010000:3.0:1789302897.706452:0:6695:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb atime: 0 -> 1789302897 00002000:00010000:3.0:1789302897.706454:0:6695:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x20:0x0] updating lvb ctime: 0 -> 1789302897 00000100:00100000:3.0:1789302897.706477:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b08a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371182208:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302897.706543:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe43095184ffea lrc: 1/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a119d expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.706564:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe43095184ffea lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a119d expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.706577:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00000100:00100000:3.0:1789302897.706624:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350550144:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3516us (3698us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302897.706633:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 438 00000100:00100000:0.0:1789302897.709814:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.709935:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350550272 00000100:00100000:3.0:1789302897.709945:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350550272 00000100:00100000:3.0:1789302897.710000:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1550 00000100:00100000:3.0:1789302897.710010:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33477:x1876219350550272:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302897.710038:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.710056:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fff1 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11a4 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.710118:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.710132:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.710142:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302897.710175:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:3.0:1789302897.710179:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302897.710195:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.710218:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe43095184fff1 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 114 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.710236:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 113 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.710251:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.710254:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1789302897.710257:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302897.710303:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33477:x1876219350550272:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 295us (497us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.710309:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1550 00000100:00100000:3.0:1789302897.713814:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.713889:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350550400 00000100:00100000:2.0:1789302897.713900:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350550400 00000100:00100000:2.0:1789302897.713955:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:2.0:1789302897.713964:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350550400:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302897.713980:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.713998:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007b025dc8/0x64fe43095184ffff lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11ab expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.714061:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371182336:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302897.714134:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302897.716840:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef31880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371182336:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302897.716879:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007b025dc8/0x64fe43095184ffff lrc: 1/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ab expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.716901:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007b025dc8/0x64fe43095184ffff lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ab expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.716915:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030000, rc 0) 00000100:00100000:2.0:1789302897.716971:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350550400:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3008us (3164us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.716980:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 439 00000100:00100000:3.0:1789302897.725097:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.725134:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350550656 00000100:00100000:3.0:1789302897.725143:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350550656 00000100:00100000:3.0:1789302897.725192:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1551 00000100:00100000:3.0:1789302897.725202:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33477:x1876219350550656:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302897.726004:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].48, 136 off14720 00000100:00100000:3.0:1789302897.726134:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33477:x1876219350550656:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 931us (1045us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.726143:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1551 00000100:00100000:2.0:1789302897.738472:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.738529:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350550784 00000100:00100000:2.0:1789302897.738539:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350550784 00000100:00100000:2.0:1789302897.738587:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1552 00000100:00100000:2.0:1789302897.738597:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33478:x1876219350550784:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302897.738620:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.738640:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe430951850006 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11b2 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.738887:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.738900:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.738912:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.738926:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.738932:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.738939:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739110:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739118:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739124:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739130:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.739135:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739143:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739155:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.739158:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.739164:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739170:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe43095185000d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739182:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739189:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739194:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739200:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.739203:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739209:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739217:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f124a9b00) 00010000:00010000:2.0:1789302897.739219:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.739222:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739228:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850014 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739705:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe430951850006 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11b2 expref: 114 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739718:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.739721:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.739725:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe430951850006 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11b2 expref: 114 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.739735:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030000, rc 0) 00000100:00100000:2.0:1789302897.739879:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33478:x1876219350550784:12345-192.168.203.11@tcp:101:bash.0 Request processed in 1283us (1414us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.739887:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1552 00000100:00100000:2.0:1789302897.756835:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.756921:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350551040 00000100:00100000:3.0:1789302897.756931:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350551040 00000100:00100000:3.0:1789302897.756993:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 79 00000100:00100000:3.0:1789302897.757005:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:2346:x1876219350551040:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:3.0:1789302897.757052:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.757073:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185001b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11c0 expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757131:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757147:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757159:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757177:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757185:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757192:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757372:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757392:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757399:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757405:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.757409:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757416:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757428:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.757432:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.757437:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757444:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850022 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757458:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757465:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757471:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757476:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.757479:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757485:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757492:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x36fdbbdd (ffff960f0cdf2600) 00010000:00010000:3.0:1789302897.757495:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.757498:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757566:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850029 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757599:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185001b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11c0 expref: 64 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757612:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.757615:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302897.757619:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185001b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11c0 expref: 64 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.757628:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc -2) 00000100:00100000:3.0:1789302897.757698:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:2346:x1876219350551040:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 694us (871us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302897.757707:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 79 00000100:00100000:3.0:1789302897.760959:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.761339:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350550912 00000100:00100000:2.0:1789302897.761350:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350550912 00000100:00100000:2.0:1789302897.761410:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1553 00000100:00100000:2.0:1789302897.761420:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33479:x1876219350550912:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302897.761439:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.761458:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850030 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11b9 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761556:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761576:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761587:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761602:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761610:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761616:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761796:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761805:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761812:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761818:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.761822:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761829:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761841:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.761845:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.761850:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761857:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663800/0x64fe430951850037 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761870:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761876:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761882:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761886:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.761889:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761895:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761903:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f124a9b00) 00010000:00010000:2.0:1789302897.761905:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.761908:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761913:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185003e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761945:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850030 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11b9 expref: 64 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761957:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.761959:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.761963:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe430951850030 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11b9 expref: 64 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.761972:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01663000, rc 0) 00000100:00100000:2.0:1789302897.762044:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33479:x1876219350550912:12345-192.168.203.11@tcp:101:rm.0 Request processed in 625us (1092us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.762050:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1553 00000100:00100000:0.0:1789302897.768795:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.768903:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350551168 00000100:00100000:3.0:1789302897.768913:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350551168 00000100:00100000:3.0:1789302897.768960:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1554 00000100:00100000:3.0:1789302897.768970:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33479:x1876219350551168:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302897.768989:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.769008:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850045 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11c7 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.769152:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.769167:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.769178:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.769196:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.769203:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.769210:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771845:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771862:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771870:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771876:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.771880:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771888:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771904:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.771907:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.771911:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771917:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185004c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771932:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771939:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771944:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771950:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.771953:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771959:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771966:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f0cdf2400) 00010000:00010000:3.0:1789302897.771969:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.771971:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.771976:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850053 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.772025:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850045 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11c7 expref: 64 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.772036:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.772038:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.772042:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850045 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11c7 expref: 64 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.772052:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:3.0:1789302897.772135:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33479:x1876219350551168:12345-192.168.203.11@tcp:101:rm.0 Request processed in 3165us (3347us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302897.772144:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1554 00000001:00080000:1.0:1789302897.773981:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968345 is committed 00000001:00080000:1.0:1789302897.774000:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968337 is committed 00000020:00080000:1.0:1789302897.774019:0:5865:0:(update_trans.c:1349:distribute_txn_batchid_cb()) lustre-MDT0000: 35 batchid updated 00000020:00080000:0.0:1789302897.774083:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 35 00000020:00080000:0.0:1789302897.774094:0:5876:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 35 00000100:00100000:2.0:1789302897.785171:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.785352:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350551296 00000100:00100000:3.0:1789302897.785362:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350551296 00000100:00100000:3.0:1789302897.785467:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1555 00000100:00100000:3.0:1789302897.785476:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:33480:x1876219350551296:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302897.785494:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.785534:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185005a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11ce expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785595:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785608:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785619:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785633:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785641:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785647:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785824:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785833:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785839:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785846:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.785850:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785857:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785869:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.785872:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.785877:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785882:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe430951850061 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785894:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785899:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785905:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785911:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.785913:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785919:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785926:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f0cdf2400) 00010000:00010000:3.0:1789302897.785928:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.785931:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785937:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850068 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785964:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185005a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ce expref: 56 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785973:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302897.785975:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.785979:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185005a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ce expref: 56 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.785987:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:3.0:1789302897.786166:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:33480:x1876219350551296:12345-192.168.203.11@tcp:101:cp.0 Request processed in 691us (1002us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302897.786174:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1555 00000100:00100000:1.0:1789302897.788858:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.788933:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350551424 00000100:00100000:2.0:1789302897.788942:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350551424 00000100:00100000:2.0:1789302897.788992:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1556 00000100:00100000:2.0:1789302897.789000:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:33480:x1876219350551424:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302897.789018:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.789043:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185006f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11d5 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789101:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789113:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789124:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789142:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789150:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789157:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789591:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789600:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789605:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789613:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.789617:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789624:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789636:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.789639:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.789644:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789650:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030800/0x64fe430951850076 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789661:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789667:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789672:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789677:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.789679:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789684:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789689:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f124a9b00) 00010000:00010000:2.0:1789302897.789691:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.789694:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789699:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01663000/0x64fe43095185007d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789728:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185006f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11d5 expref: 56 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789736:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.789738:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.789742:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185006f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11d5 expref: 56 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.789750:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030000, rc 0) 00000100:00100000:2.0:1789302897.789802:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:33480:x1876219350551424:12345-192.168.203.11@tcp:101:cp.0 Request processed in 803us (951us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.789809:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1556 00000100:00100000:1.0:1789302897.793125:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.793230:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350551680 00000100:00100000:3.0:1789302897.793239:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350551680 00000100:00100000:3.0:1789302897.793304:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1557 00000100:00100000:3.0:1789302897.793313:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:33480:x1876219350551680:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302897.793332:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.793352:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850084 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11dc expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.793893:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.793909:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 111 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.793918:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.793947:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 111 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.793960:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 111 pid: 6919 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.793974:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371182464:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302897.794124:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302897.799193:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.799252:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350551808 00000100:00100000:0.0:1789302897.799268:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350551808 00010000:00010000:0.0:1789302897.799305:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 112 pid: 6919 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.799343:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00100000:0.0:1789302897.799352:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2347:x1876219350551808:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302897.799364:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302897.799371:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 112 pid: 6919 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.799395:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 112 pid: 6919 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.799414:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.799420:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.799430:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.799443:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.799471:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302897.799537:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2347:x1876219350551808:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 186us (353us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.799544:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 452 00010000:00010000:3.0:1789302897.800829:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe43095184fff8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11a4 expref: 111 pid: 6919 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.800849:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa6d80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371182464:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302897.800871:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.800956:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.800963:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.800989:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.801001:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.801010:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302897.802117:0:5845:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x20:0x0], index 1: 0 00010000:00010000:2.0:1789302897.802769:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802782:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802791:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802823:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802831:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802838:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802844:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302897.802849:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802855:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802867:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302897.802870:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302897.802875:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802881:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185008b lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302897.802895:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000895a6c55 reply state 000000000134617c transno 4294968346 ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.802903:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302897.802934:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850084 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 58 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802953:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 57 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.802960:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:2.0:1789302897.802962:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302897.802965:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120dda00, rc 0) 00000100:00100000:2.0:1789302897.803086:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:33480:x1876219350551680:12345-192.168.203.11@tcp:101:cp.0 Request processed in 9773us (9968us total) trans 4294968346 rc 0/0 00000100:00100000:2.0:1789302897.803095:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1557 00000040:00080000:0.0:1789302897.804672:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302897.804679:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302897.804716:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 48 count 7 00000040:00080000:0.0:1789302897.804721:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:0.0:1789302897.805033:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371182592:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302897.805119:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2180 going to sleep for 11 seconds 00000100:00100000:2.0:1789302897.814170:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.814229:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.814264:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350552064 00000100:00100000:2.0:1789302897.814271:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350552064 00000100:00100000:2.0:1789302897.814317:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1558 00000100:00100000:2.0:1789302897.814326:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:33481:x1876219350552064:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302897.814358:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2e:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302897.814470:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:33481:x1876219350552064:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 145us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.814478:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1558 00000100:00100000:2.0:1789302897.814538:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.814643:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350551936 00000100:00100000:0.0:1789302897.814648:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350551936 00000100:00100000:0.0:1789302897.814677:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 440 00000100:00100000:0.0:1789302897.814684:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350551936:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302897.814694:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.814715:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11e3 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302897.814739:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x20:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:0.0:1789302897.814756:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789302897.814760:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302897.814770:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 14 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.814793:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 14 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.814805:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:0.0:1789302897.814830:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2347:x1876219350551936:12345-192.168.203.11@tcp:101:cp.0 Request processed in 147us (668us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.814837:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 440 00000100:00100000:1.0:1789302897.815008:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371182592:192.168.203.11@tcp:107:hsm_cdtr.0 00010000:00010000:3.0:1789302897.815314:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.815324:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.815332:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.815338:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.815342:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.815350:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.815363:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f124a9500) 00010000:00010000:3.0:1789302897.815367:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302897.815371:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.815388:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850092 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.819132:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.819408:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350552192 00000100:00100000:1.0:1789302897.819417:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350552192 00000100:00100000:1.0:1789302897.819476:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350552320 00000100:00100000:1.0:1789302897.819479:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350552320 00000100:00100000:1.0:1789302897.819515:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1559 00000100:00100000:1.0:1789302897.819524:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:33483:x1876219350552192:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302897.819552:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x30:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302897.819636:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:33483:x1876219350552192:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 112us (5096us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.819644:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1559 00000100:00100000:1.0:1789302897.819660:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1560 00000100:00100000:1.0:1789302897.819667:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33482:x1876219350552320:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302897.819677:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2f:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302897.819708:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33482:x1876219350552320:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 42us (585us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.819714:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1560 00000100:00100000:3.0:1789302897.819983:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.820112:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350552576 00000100:00100000:1.0:1789302897.820116:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350552576 00000100:00100000:1.0:1789302897.820146:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1561 00000100:00100000:1.0:1789302897.820153:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:33481:x1876219350552576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302897.820162:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302897.820180:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518500ae lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11ea expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.820255:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe4309518500b5 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.820268:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe4309518500b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.820278:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe4309518500b5 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302897.820288:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2e:0x0]: returning size 158 00010000:00010000:1.0:1789302897.820418:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518500ae lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ea expref: 60 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.820436:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe4309518500b5 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ea expref: 59 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.820444:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302897.820563:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302897.820567:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8c00, rc 0) 00000100:00100000:1.0:1789302897.820598:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:33481:x1876219350552576:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 446us (615us total) trans 4294968347 rc 0/0 00000100:00100000:1.0:1789302897.820605:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1561 00000100:00100000:0.0:1789302897.823635:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.823711:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350552704 00000100:00100000:3.0:1789302897.823720:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350552704 00000100:00100000:3.0:1789302897.823765:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:3.0:1789302897.823773:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:33480:x1876219350552704:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302897.823790:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.823807:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 2/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11f1 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.823836:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 15 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.823845:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302897.823849:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302897.823855:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 15 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.823885:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 15 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.823898:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 15 pid: 8571 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.823912:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371182720:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302897.823985:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a900 going to sleep for 11 seconds 00000100:00100000:0.0:1789302897.827849:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.827912:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.827969:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350552960 00000100:00100000:1.0:1789302897.827979:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350552960 00000100:00100000:1.0:1789302897.828045:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 71 00000100:00100000:1.0:1789302897.828054:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd06050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:2348:x1876219350552960:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.828071:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.828093:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe4309518500b5 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ea expref: 59 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.828110:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302897.828115:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302897.828160:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd06050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:2348:x1876219350552960:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 108us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.828167:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 71 00000100:00100000:3.0:1789302897.834804:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350552832 00000100:00100000:3.0:1789302897.834820:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350552832 00000100:00100000:1.0:1789302897.835194:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.835222:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.835239:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350553088 00000100:00100000:1.0:1789302897.835242:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350553088 00010000:00010000:1.0:1789302897.835284:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 17 pid: 8571 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.835319:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 453 00000100:00100000:1.0:1789302897.835328:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350553088:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.835341:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.835363:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 17 pid: 8571 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.835385:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 17 pid: 8571 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.835410:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302897.835414:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302897.835422:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 17 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.835432:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 17 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.835471:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 17 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.835484:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371182848:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302897.835614:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e2e9d08/0x64fe4309518500a7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11e3 expref: 17 pid: 8571 timeout: 834 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.835631:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371182720:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302897.835653:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.835663:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf1a00, rc 0) 00000100:00100000:2.0:1789302897.836797:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 442 00000100:00100000:2.0:1789302897.836807:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2346:x1876219350552832:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302897.836842:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.836863:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 16 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.836871:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1e:0x0].0x0 since=1s v=(0 0) 00010000:00010000:2.0:1789302897.836875:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302897.836878:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.836904:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 16 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.836918:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 16 pid: 18578 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.836930:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371182976:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302897.837139:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80300 going to sleep for 11 seconds 00000100:00100000:3.0:1789302897.837227:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:33480:x1876219350552704:12345-192.168.203.11@tcp:101:cp.0 Request processed in 13453us (13599us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.837235:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 441 00000100:00100000:2.0:1789302897.837357:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b900 going to sleep for 11 seconds 00000100:00100000:2.0:1789302897.837418:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.837868:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350553344 00000100:00100000:3.0:1789302897.837875:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350553344 00000100:00100000:3.0:1789302897.837903:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350553600 00000100:00100000:3.0:1789302897.837906:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350553600 00000100:00100000:3.0:1789302897.837926:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1562 00000100:00100000:3.0:1789302897.837933:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+59:33482:x1876219350553344:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302897.837946:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.837962:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe4309518500d1 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11f8 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838060:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518500d8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838074:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518500d8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838084:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518500d8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302897.838093:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x2f:0x0]: returning size 158 00010000:00010000:3.0:1789302897.838210:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe4309518500d1 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f8 expref: 62 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838233:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518500d8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f8 expref: 61 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838241:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302897.838244:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302897.838247:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0600, rc 0) 00000100:00100000:3.0:1789302897.838290:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:33482:x1876219350553344:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 358us (3101us total) trans 4294968348 rc 0/0 00000100:00100000:3.0:1789302897.838297:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1562 00000100:00100000:3.0:1789302897.838310:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1563 00000100:00100000:3.0:1789302897.838316:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+60:33483:x1876219350553600:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302897.838324:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.838331:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518500e6 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a11ff expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838368:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518500ed lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838390:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518500ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838398:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518500ed lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302897.838405:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x30:0x0]: returning size 158 00010000:00010000:3.0:1789302897.838463:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518500e6 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ff expref: 63 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838482:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518500ed lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ff expref: 62 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.838488:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:3.0:1789302897.838491:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302897.838493:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:3.0:1789302897.838531:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33483:x1876219350553600:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 215us (1114us total) trans 4294968349 rc 0/0 00000100:00100000:3.0:1789302897.838538:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1563 00000100:00100000:2.0:1789302897.842728:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c4700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371182976:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302897.842761:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.842854:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.842879:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350553856 00000100:00100000:2.0:1789302897.842885:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350553856 00000100:00100000:2.0:1789302897.842928:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:2.0:1789302897.842936:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33480:x1876219350553856:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302897.843241:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33480:x1876219350553856:12345-192.168.203.11@tcp:35:cp.0 Request processed in 305us (387us total) trans 4294968350 rc 0/0 00000100:00100000:2.0:1789302897.843249:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 330 00000100:00100000:2.0:1789302897.843303:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.843970:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbaa00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371182848:192.168.203.11@tcp:105: 00010000:00010000:3.0:1789302897.843985:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302897.844010:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2348:x1876219350553088:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 8683us (8787us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.844018:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 453 00000100:00100000:3.0:1789302897.844045:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350553984 00000100:00100000:3.0:1789302897.844049:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350553984 00010000:00010000:3.0:1789302897.844068:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 15 pid: 18578 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.844090:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 454 00000100:00100000:3.0:1789302897.844096:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2347:x1876219350553984:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302897.844106:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302897.844124:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 15 pid: 18578 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.844141:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 15 pid: 18578 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.844162:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000497b64b7/0x64fe43095184fcfd lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fe4 expref: 15 pid: 18578 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.844177:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302897.844181:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302897.844187:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.844197:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.844214:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302897.844230:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2347:x1876219350553984:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 134us (928us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.844237:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 454 00010000:00010000:2.0:1789302897.844895:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844906:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844930:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844938:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844947:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844955:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01663800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302897.844959:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844968:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844981:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.844989:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000040326cc/0x64fe4309518500ca lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.849689:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.849755:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.849828:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.849848:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.850119:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2346:x1876219350552832:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 13310us (22275us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.850130:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 442 00000100:00100000:2.0:1789302897.850158:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350554112 00000100:00100000:2.0:1789302897.850165:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350554112 00000100:00100000:2.0:1789302897.850202:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350554240 00000100:00100000:2.0:1789302897.850205:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350554240 00000100:00100000:2.0:1789302897.850224:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 443 00000100:00100000:2.0:1789302897.850231:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350554112:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302897.850281:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.850302:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.850310:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302897.850315:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302897.850320:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 4/1,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.850356:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 13 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.850370:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 13 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.850395:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371183104:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302897.851275:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b900 going to sleep for 11 seconds 00000100:00100000:1.0:1789302897.851387:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350554496 00000100:00100000:1.0:1789302897.851394:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350554496 00000100:00100000:1.0:1789302897.851431:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350554368 00000100:00100000:1.0:1789302897.851434:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350554368 00000100:00100000:1.0:1789302897.851454:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 72 00000100:00100000:1.0:1789302897.851462:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:2348:x1876219350554496:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.851473:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.851494:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518500d8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f8 expref: 64 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.851528:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302897.851533:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302897.851560:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:2348:x1876219350554496:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 98us (1731us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.851566:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 72 00000100:00100000:1.0:1789302897.851579:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 73 00000100:00100000:1.0:1789302897.851584:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:2348:x1876219350554368:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.851591:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.851600:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518500ed lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11ff expref: 62 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.851609:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:1.0:1789302897.851613:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302897.851627:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:2348:x1876219350554368:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 43us (1779us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.851633:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 73 00000100:00100000:1.0:1789302897.861214:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 444 00000100:00100000:1.0:1789302897.861239:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350554240:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302897.861304:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.861325:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.861341:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x1f:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302897.861345:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302897.861350:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.861395:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 11 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.861408:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 11 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.861422:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371183232:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302897.871178:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3fe80cc0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302897.876194:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.876295:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.877253:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c7b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371183104:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302897.877298:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.877444:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fba300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371183232:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302897.877460:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.880145:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.880246:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350555008 00000100:00100000:2.0:1789302897.880255:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350555008 00000100:00100000:2.0:1789302897.880314:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1564 00000100:00100000:2.0:1789302897.880324:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:33481:x1876219350555008:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302897.880433:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:33481:x1876219350555008:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 110us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.880441:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1564 00000100:00100000:1.0:1789302897.883828:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350554624 00000100:00100000:1.0:1789302897.883841:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350554624 00002000:00100000:1.0:1789302897.883891:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1f:0x0] (0->157) req@ffff960e02b09180 x1876219350554624/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 648/0 e 0 to 0 dl 1789302908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302897.883902:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02b09180 with x1876219350554624 ext(0->157) 00010000:00010000:1.0:1789302897.883911:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 14 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.883927:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 14 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302897.883941:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960e02b09180 00000100:00100000:1.0:1789302897.883967:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350554752 00000100:00100000:1.0:1789302897.883969:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350554752 00002000:00100000:1.0:1789302897.883984:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1f:0x0] (0->157) req@ffff960f3ad47800 x1876219350554752/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 648/0 e 0 to 0 dl 1789302908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302897.883992:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad47800 with x1876219350554752 ext(0->157) 00010000:00010000:1.0:1789302897.883997:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 12 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.884007:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 12 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302897.884014:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3ad47800 00000100:00100000:1.0:1789302897.884040:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 14 00000100:00100000:1.0:1789302897.884048:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2345:x1876219350554624:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:0.0:1789302897.884552:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 15 00000100:00100000:0.0:1789302897.884563:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2346:x1876219350554752:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302897.891960:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2345:x1876219350554624:12345-192.168.203.11@tcp:4:cp.0 Request processed in 7900us (15762us total) trans 4294967383 rc 0/0 00000100:00100000:1.0:1789302897.891976:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 14 00002000:00100000:1.0:1789302897.891989:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x1f:0x0] (0->157) req@ffff960e02b09180 x1876219350554624/t4294967383(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 648/448 e 0 to 0 dl 1789302908 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302897.892000:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02b09180 with x1876219350554624 ext(0->157) 00010000:00010000:1.0:1789302897.892011:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 16 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.892045:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 16 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302897.892054:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960e02b09180 00000100:00100000:0.0:1789302897.892323:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2346:x1876219350554752:12345-192.168.203.11@tcp:4:cp.0 Request processed in 7761us (16026us total) trans 4294967385 rc 0/0 00000100:00100000:0.0:1789302897.892332:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 15 00002000:00100000:0.0:1789302897.892338:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x1f:0x0] (0->157) req@ffff960f3ad47800 x1876219350554752/t4294967385(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 648/448 e 0 to 0 dl 1789302908 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302897.892347:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ad47800 with x1876219350554752 ext(0->157) 00010000:00010000:0.0:1789302897.892354:0:6700:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 14 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.892365:0:6700:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 14 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302897.892372:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3ad47800 00000100:00100000:2.0:1789302897.897357:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.900648:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350555136 00000100:00100000:0.0:1789302897.900661:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350555136 00000100:00100000:0.0:1789302897.900730:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 445 00000100:00100000:0.0:1789302897.900739:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350555136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302897.900756:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.900777:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1206 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302897.900804:0:6696:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1e:0x0] initial LVB size: 158, mtime: 0x6aa69870, atime: 0x0, ctime: 0x6aa69870, blocks: 0x1 00010000:00010000:0.0:1789302897.900822:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=734s v=(0 0) 00010000:00010000:0.0:1789302897.900826:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302897.900834:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.900850:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.900862:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345bc00, rc 0) 00000100:00100000:0.0:1789302897.901372:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2347:x1876219350555136:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 632us (4021us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.901393:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 445 00000100:00100000:1.0:1789302897.903978:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.905862:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350555264 00000100:00100000:0.0:1789302897.905874:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350555264 00010000:00010000:0.0:1789302897.905915:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 14 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.905963:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 455 00000100:00100000:0.0:1789302897.905972:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2348:x1876219350555264:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302897.905985:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302897.906005:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 14 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.906018:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 14 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.906048:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe43095184fdcf lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1038 expref: 14 pid: 6697 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.906068:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302897.906072:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302897.906078:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.906088:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.906119:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302897.906160:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350555264:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 189us (2190us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.906168:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 455 00000100:00100000:1.0:1789302897.907115:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.907225:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350555392 00000100:00100000:0.0:1789302897.907231:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350555392 00010000:00010000:0.0:1789302897.907253:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 17 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.907294:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:0.0:1789302897.907302:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2347:x1876219350555392:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302897.907312:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302897.907328:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 17 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.907340:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 17 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.907356:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004c7ddea/0x64fe43095184fea1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a108c expref: 17 pid: 6696 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.907368:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302897.907372:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302897.907389:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.907403:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.907428:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302897.907538:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907548:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907573:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907581:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907589:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907597:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7e000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302897.907601:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907609:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907621:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.907628:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000336579d8/0x64fe4309518500fb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.907677:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350554112:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 57446us (57996us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.907689:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 443 00000100:00100000:0.0:1789302897.908676:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2347:x1876219350555392:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1372us (1561us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.908686:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 456 00010000:00010000:2.0:1789302897.909554:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909568:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909597:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909605:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909614:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909620:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302897.909625:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909634:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909650:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.909663:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe430951850102 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.909734:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350554240:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 48506us (59976us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.909742:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 444 00000100:00100000:3.0:1789302897.910891:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.910922:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.910984:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350555520 00000100:00100000:0.0:1789302897.910992:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350555520 00000100:00100000:0.0:1789302897.911024:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350555776 00000100:00100000:0.0:1789302897.911036:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350555776 00000100:00100000:0.0:1789302897.911062:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1565 00000100:00100000:0.0:1789302897.911070:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33486:x1876219350555520:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.911086:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.911101:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850110 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a120d expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911155:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911168:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911177:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911191:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911199:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911205:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911294:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850125 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911303:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850125 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911310:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850125 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302897.911339:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x32:0x0]: returning size 158 00000004:00200000:0.0:1789302897.911343:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302897.911356:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850125 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911365:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850125 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911384:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911391:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911397:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911407:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.911412:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911419:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911429:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302897.911432:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.911437:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911443:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850117 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911463:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911470:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911476:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911481:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302897.911484:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911489:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911500:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f0cf83300) 00010000:00010000:0.0:1789302897.911519:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302897.911522:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911529:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185011e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911555:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850110 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a120d expref: 113 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911567:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850125 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a120d expref: 112 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.911575:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:0.0:1789302897.911577:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302897.911580:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a400, rc 0) 00000100:00100000:1.0:1789302897.911704:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1566 00000100:00100000:1.0:1789302897.911714:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:33481:x1876219350555776:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302897.913749:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33486:x1876219350555520:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2671us (2852us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.913759:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1565 00000100:00100000:1.0:1789302897.913950:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:33481:x1876219350555776:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 2237us (3026us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.913959:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1566 00000100:00100000:1.0:1789302897.919968:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.920019:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.920054:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.920112:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350556032 00000100:00100000:2.0:1789302897.920123:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350556032 00000100:00100000:2.0:1789302897.920172:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350556288 00000100:00100000:2.0:1789302897.920175:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350556288 00000100:00100000:2.0:1789302897.920199:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1567 00000100:00100000:1.0:1789302897.920209:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1568 00000100:00100000:2.0:1789302897.920211:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33483:x1876219350556032:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302897.920223:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33482:x1876219350556288:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302897.920320:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33482:x1876219350556288:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 102us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.920328:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1568 00000100:00100000:1.0:1789302897.923603:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.923734:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.926019:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:33483:x1876219350556032:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 5802us (6050us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.926041:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1567 00000100:00100000:2.0:1789302897.926095:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350556672 00000100:00100000:2.0:1789302897.926102:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350556672 00000100:00100000:2.0:1789302897.926145:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:2.0:1789302897.926153:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350556672:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302897.926168:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.926185:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007b025dc8/0x64fe43095185012c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a124c expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.926238:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371183360:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302897.926562:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350556544 00000100:00100000:1.0:1789302897.926567:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350556544 00000100:00100000:1.0:1789302897.926604:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 80 00000100:00100000:1.0:1789302897.926618:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2346:x1876219350556544:12345-192.168.203.11@tcp:63:ll_sa.0 00000100:00100000:2.0:1789302897.926666:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:0.0:1789302897.926735:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:1.0:1789302897.926798:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302897.926813:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850133 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1214 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926863:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926877:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926889:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926904:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926912:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926919:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926969:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926977:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926985:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.926996:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302897.927000:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927009:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927021:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302897.927024:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302897.927029:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927036:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185013a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927055:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927062:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927069:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927075:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302897.927078:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927084:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927095:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f2c94d900) 00010000:00010000:1.0:1789302897.927098:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302897.927101:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927107:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe430951850141 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927133:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850133 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1214 expref: 113 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927144:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302897.927146:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302897.927150:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850133 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1214 expref: 113 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.927159:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc -2) 00000100:00100000:1.0:1789302897.927194:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:2346:x1876219350556544:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 577us (3598us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302897.927202:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 80 00000100:00100000:2.0:1789302897.929730:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350556800 00000100:00100000:2.0:1789302897.929741:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350556800 00000100:00100000:2.0:1789302897.929803:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00100000:2.0:1789302897.929813:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2347:x1876219350556800:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302897.929828:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.929846:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1253 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302897.929870:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x1f:0x0] initial LVB size: 158, mtime: 0x6aa69870, atime: 0x0, ctime: 0x6aa69870, blocks: 0x1 00010000:00010000:2.0:1789302897.929886:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789302897.929891:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302897.929899:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 13 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.929914:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 13 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.929927:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00031c00, rc 0) 00000100:00100000:2.0:1789302897.929968:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350556800:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 157us (3233us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.929975:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 447 00000100:00100000:3.0:1789302897.930163:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350556416 00000100:00100000:3.0:1789302897.930168:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350556416 00002000:00100000:3.0:1789302897.930198:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1e:0x0] (0->4095) req@ffff960f25fba300 x1876219350556416/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 488/0 e 0 to 0 dl 1789302908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302897.930207:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fba300 with x1876219350556416 ext(0->4095) 00002000:00010000:3.0:1789302897.930212:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25fba300 00000100:00100000:1.0:1789302897.930689:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.930787:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350556928 00000100:00100000:0.0:1789302897.930792:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350556928 00000100:00100000:3.0:1789302897.932137:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 129 00000100:00100000:3.0:1789302897.932152:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2345:x1876219350556416:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302897.932612:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2345:x1876219350556416:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 177us (12268us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302897.932623:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 129 00002000:00100000:3.0:1789302897.932633:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x1e:0x0] (0->4095) req@ffff960f25fba300 x1876219350556416/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 488/4536 e 0 to 0 dl 1789302908 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302897.932704:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25fba300 with x1876219350556416 ext(0->4095) 00002000:00010000:3.0:1789302897.932710:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25fba300 00002000:00010000:2.0:1789302897.934369:0:13439:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302897.934384:0:13439:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb mtime: 0 -> 1789302897 00002000:00010000:2.0:1789302897.934387:0:13439:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb atime: 0 -> 1789302897 00002000:00010000:2.0:1789302897.934389:0:13439:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x20:0x0] updating lvb ctime: 0 -> 1789302897 00000100:00100000:2.0:1789302897.934408:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c5c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371183360:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302897.934444:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007b025dc8/0x64fe43095185012c lrc: 1/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a124c expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.934586:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007b025dc8/0x64fe43095185012c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a124c expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.934599:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030000, rc 0) 00000100:00100000:2.0:1789302897.934718:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350556672:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 8564us (10983us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302897.934726:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 446 00000100:00100000:2.0:1789302897.934752:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 448 00000100:00100000:2.0:1789302897.934762:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f08599c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2348:x1876219350556928:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302897.934773:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302897.934791:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a125a expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302897.934808:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x1f:0x0] initial LVB size: 158, mtime: 0x6aa69870, atime: 0x0, ctime: 0x6aa69870, blocks: 0x1 00010000:00010000:2.0:1789302897.934824:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=734s v=(0 0) 00010000:00010000:2.0:1789302897.934828:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302897.934835:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 16 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.935276:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.935303:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.935323:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350557184 00000100:00100000:3.0:1789302897.935329:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350557184 00000100:00100000:3.0:1789302897.935357:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350557312 00000100:00100000:3.0:1789302897.935359:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350557312 00000100:00100000:3.0:1789302897.935386:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1569 00000100:00100000:3.0:1789302897.935394:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33482:x1876219350557184:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302897.935406:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 16 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302897.935425:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030800, rc 0) 00000100:00100000:3.0:1789302897.935470:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33482:x1876219350557184:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 76us (194us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.935477:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1569 00000100:00100000:3.0:1789302897.935490:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1570 00000100:00100000:3.0:1789302897.935496:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+61:33481:x1876219350557312:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302897.935522:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.935535:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850156 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1261 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.935559:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.935572:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.935582:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.935661:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850156 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 63 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.935676:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 62 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.935683:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302897.935686:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302897.935690:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:3.0:1789302897.935712:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33481:x1876219350557312:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 216us (408us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.935718:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1570 00000100:00100000:2.0:1789302897.936180:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f08599c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2348:x1876219350556928:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1419us (5490us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302897.936187:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 448 00000100:00100000:3.0:1789302897.938769:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.940665:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350557440 00000100:00100000:1.0:1789302897.940677:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350557440 00000100:00100000:1.0:1789302897.940744:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1571 00000100:00100000:1.0:1789302897.940755:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33481:x1876219350557440:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302897.940786:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2e:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302897.940893:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+62:33481:x1876219350557440:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 140us (2131us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.940901:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1571 00000100:00100000:3.0:1789302897.941213:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.941296:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.941341:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350557568 00000100:00100000:1.0:1789302897.941346:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350557568 00000100:00100000:0.0:1789302897.941354:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350557696 00000100:00100000:0.0:1789302897.941361:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350557696 00000100:00100000:1.0:1789302897.941395:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1572 00002000:00100000:0.0:1789302897.941405:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1f:0x0] (0->4095) req@ffff960f25c45f80 x1876219350557696/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 488/0 e 0 to 0 dl 1789302908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00000100:00100000:1.0:1789302897.941407:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33486:x1876219350557568:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302897.941422:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00002000:00100000:0.0:1789302897.941429:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25c45f80 with x1876219350557696 ext(0->4095) 00002000:00010000:0.0:1789302897.941435:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f25c45f80 00010000:00010000:1.0:1789302897.941443:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850164 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1268 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.941460:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 130 00000100:00100000:0.0:1789302897.941468:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350557696:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00010000:00010000:1.0:1789302897.941523:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.941538:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.941551:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302897.941572:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:1.0:1789302897.941576:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302897.941590:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.941615:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850164 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 114 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.941635:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 113 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.941645:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350557696:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 175us (346us total) trans 0 rc 4096/4096 00010000:00010000:1.0:1789302897.941660:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00000100:00100000:0.0:1789302897.941661:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 130 00010000:00010000:1.0:1789302897.941667:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00002000:00100000:0.0:1789302897.941671:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x1f:0x0] (0->4095) req@ffff960f25c45f80 x1876219350557696/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 488/4536 e 0 to 0 dl 1789302908 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00010000:00010000:1.0:1789302897.941672:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9400, rc 0) 00002000:00100000:0.0:1789302897.941684:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25c45f80 with x1876219350557696 ext(0->4095) 00002000:00010000:0.0:1789302897.941688:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f25c45f80 00000100:00100000:1.0:1789302897.941708:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33486:x1876219350557568:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 304us (495us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.941716:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1572 00000100:00100000:1.0:1789302897.948112:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.948493:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.948544:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.948589:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350558080 00000100:00100000:0.0:1789302897.948598:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350558080 00000100:00100000:0.0:1789302897.948650:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 449 00000100:00100000:0.0:1789302897.948660:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350558080:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302897.948699:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948716:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302897.948720:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302897.948726:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948739:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948771:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948780:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948787:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948795:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302897.948799:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948808:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948823:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.948830:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe430951850172 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.948891:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350558080:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 233us (398us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.948899:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 449 00000100:00100000:1.0:1789302897.950897:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.950928:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.950986:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350558208 00000100:00100000:0.0:1789302897.950994:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350558208 00000100:00100000:0.0:1789302897.951044:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 450 00000100:00100000:0.0:1789302897.951053:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350558208:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302897.951067:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302897.951082:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000df8c373b/0x64fe430951850179 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a126f expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.951133:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371183488:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302897.951193:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2cc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302897.951223:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350557952 00000100:00100000:2.0:1789302897.951227:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350557952 00000100:00100000:2.0:1789302897.951347:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350558336 00000100:00100000:2.0:1789302897.951351:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350558336 00000100:00100000:3.0:1789302897.951605:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350558592 00000100:00100000:3.0:1789302897.951609:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350558592 00000100:00100000:3.0:1789302897.951632:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1573 00000100:00100000:3.0:1789302897.951639:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:33483:x1876219350557952:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302897.951695:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:33483:x1876219350557952:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 57us (3592us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.951702:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1573 00000100:00100000:3.0:1789302897.951718:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1574 00000100:00100000:3.0:1789302897.951724:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33482:x1876219350558336:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302897.951730:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302897.951743:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850180 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1276 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.951769:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.951782:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.951793:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.951881:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850180 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 65 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.951899:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 64 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.951907:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302897.951910:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302897.951913:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7fc00, rc 0) 00000100:00100000:3.0:1789302897.951937:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:33482:x1876219350558336:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 213us (3393us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.951943:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1574 00000100:00100000:3.0:1789302897.951956:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1575 00000100:00100000:3.0:1789302897.951961:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33481:x1876219350558592:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302897.951989:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.951999:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 63 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.952008:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.952040:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 63 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.952059:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 63 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.952069:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c46300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371183616:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302897.952099:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00aa80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302897.956331:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.956471:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c46300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371183616:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302897.956496:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.957054:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350558720 00000100:00100000:0.0:1789302897.957063:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350558720 00000100:00100000:0.0:1789302897.957107:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1576 00000100:00100000:0.0:1789302897.957115:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:33482:x1876219350558720:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302897.957155:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x2f:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302897.957254:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:33482:x1876219350558720:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 139us (930us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.957261:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1576 00000100:00100000:0.0:1789302897.957312:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267df80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371183488:192.168.203.11@tcp:106: 00010000:00010000:0.0:1789302897.957336:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000df8c373b/0x64fe430951850179 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a126f expref: 18 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.957357:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000df8c373b/0x64fe430951850179 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a126f expref: 18 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.957387:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc 0) 00000100:00100000:0.0:1789302897.957414:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350558208:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 6363us (6524us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302897.957421:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 450 00000100:00100000:3.0:1789302897.959655:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.959697:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.959762:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350558848 00000100:00100000:0.0:1789302897.959771:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350558848 00010000:00010000:0.0:1789302897.959803:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 64 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302897.959841:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:0.0:1789302897.959849:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd05c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:2348:x1876219350558848:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302897.959863:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302897.959869:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 64 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.959881:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 64 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.959900:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe43095185015d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1261 expref: 64 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.959916:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:0.0:1789302897.959922:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.959931:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.959940:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302897.959959:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302897.959993:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd05c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:2348:x1876219350558848:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 145us (346us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.960000:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:0.0:1789302897.960061:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350558976 00000100:00100000:0.0:1789302897.960064:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350558976 00002000:00100000:0.0:1789302897.960091:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1f:0x0] (0->4095) req@ffff960f25c46300 x1876219350558976/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 488/0 e 0 to 0 dl 1789302908 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302897.960101:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25c46300 with x1876219350558976 ext(0->4095) 00002000:00010000:0.0:1789302897.960105:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25c46300 00000100:00100000:0.0:1789302897.960129:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 131 00000100:00100000:0.0:1789302897.960136:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2348:x1876219350558976:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302897.960286:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2348:x1876219350558976:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 150us (588us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302897.960294:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 131 00002000:00100000:0.0:1789302897.960300:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x1f:0x0] (0->4095) req@ffff960f25c46300 x1876219350558976/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:473/0 lens 488/4536 e 0 to 0 dl 1789302908 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302897.960308:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f25c46300 with x1876219350558976 ext(0->4095) 00002000:00010000:0.0:1789302897.960312:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f25c46300 00010000:00010000:3.0:1789302897.960427:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960436:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960638:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960646:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960653:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960659:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.960664:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960672:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960682:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302897.960684:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302897.960688:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.960694:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe43095185018e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.963819:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33481:x1876219350558592:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 11848us (12880us total) trans 4294968351 rc 0/0 00000100:00100000:3.0:1789302897.963832:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1575 00000100:00100000:2.0:1789302897.967771:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.967809:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.967879:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350559360 00000100:00100000:3.0:1789302897.967888:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350559360 00000100:00100000:3.0:1789302897.967941:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:3.0:1789302897.967951:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350559360:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302897.967993:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968010:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302897.968014:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302897.968020:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968050:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968080:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968088:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968097:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968104:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302897.968108:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968117:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968131:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.968140:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000d9ee8de/0x64fe430951850195 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.968214:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350559360:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 264us (403us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.968221:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 451 00000100:00100000:0.0:1789302897.968323:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350559232 00000100:00100000:0.0:1789302897.968327:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350559232 00000100:00100000:0.0:1789302897.968367:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1577 00000100:00100000:0.0:1789302897.968383:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33486:x1876219350559232:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:0.0:1789302897.968733:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].49, 136 off14856 00000100:00100000:0.0:1789302897.968776:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33486:x1876219350559232:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 401us (1012us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302897.968783:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1577 00000100:00100000:2.0:1789302897.972115:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302897.972155:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302897.972268:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350559616 00000100:00100000:0.0:1789302897.972277:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350559616 00000100:00100000:0.0:1789302897.972329:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:0.0:1789302897.972339:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+63:33481:x1876219350559616:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302897.972588:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:33481:x1876219350559616:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 250us (480us total) trans 4294968352 rc 0/0 00000100:00100000:0.0:1789302897.972596:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 331 00000100:00100000:1.0:1789302897.974188:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350559744 00000100:00100000:1.0:1789302897.974199:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350559744 00000100:00100000:1.0:1789302897.974260:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1578 00000100:00100000:1.0:1789302897.974271:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+64:33483:x1876219350559744:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302897.974288:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302897.974310:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095185019c lrc: 2/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a127d expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.974343:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.974358:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.974386:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.974485:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe43095185019c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 66 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.974535:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 65 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.974545:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:1.0:1789302897.974548:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302897.974551:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc 0) 00000100:00100000:1.0:1789302897.974610:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+65:33483:x1876219350559744:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 341us (2453us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.974617:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1578 00000100:00100000:2.0:1789302897.978324:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.978409:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350560000 00000100:00100000:3.0:1789302897.978418:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350560000 00000100:00100000:3.0:1789302897.978485:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1579 00000100:00100000:3.0:1789302897.978495:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+65:33482:x1876219350560000:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302897.978595:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.978611:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 65 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.978620:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 4/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.978662:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 65 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.978676:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 65 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.978690:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371183744:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302897.978753:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00aa80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302897.983946:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302897.984022:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350560128 00000100:00100000:3.0:1789302897.984039:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350560128 00000100:00100000:3.0:1789302897.984097:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1580 00000100:00100000:3.0:1789302897.984106:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:33483:x1876219350560128:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302897.984140:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x30:0x0] from 18446744073709551615 #0 00000100:00100000:3.0:1789302897.984227:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:33483:x1876219350560128:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 122us (289us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302897.984234:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1580 00000100:00100000:3.0:1789302897.987423:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44700 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371183744:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302897.987463:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302897.989446:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.989529:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350560256 00000100:00100000:1.0:1789302897.989540:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350560256 00010000:00010000:1.0:1789302897.989577:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 66 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302897.989616:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:1.0:1789302897.989627:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:2348:x1876219350560256:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302897.989641:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302897.989648:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 66 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.989661:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 66 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.989686:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe430951850187 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1276 expref: 66 pid: 18427 timeout: 834 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.989724:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302897.989730:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.989741:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.989750:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302897.989778:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302897.989840:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+65:2348:x1876219350560256:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 214us (399us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302897.989847:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 458 00010000:00010000:3.0:1789302897.990461:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.990473:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.990793:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.991122:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.991132:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.991138:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302897.991143:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.991151:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.991165:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:3.0:1789302897.991168:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302897.991172:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 2/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302897.991179:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe4309518501aa lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302897.991318:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+65:33482:x1876219350560000:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 12821us (13000us total) trans 4294968353 rc 0/0 00000100:00100000:3.0:1789302897.991327:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1579 00000100:00100000:2.0:1789302897.991493:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302897.997470:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350560384 00000100:00100000:1.0:1789302897.997484:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350560384 00000100:00100000:1.0:1789302898.001203:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00100000:1.0:1789302898.001229:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350560384:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302898.001273:0:8571:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.001291:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302898.001295:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.001398:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.001417:0:8571:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.002752:0:8571:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.002769:0:8571:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.002779:0:8571:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.002787:0:8571:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302898.002792:0:8571:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.002801:0:8571:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.002820:0:8571:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.002827:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518501b1 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.002968:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350560384:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 1752us (11472us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.002978:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 452 00000100:00080000:1.0:1789302898.003019:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302898.004933:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.004968:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.005128:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350560640 00000100:00100000:0.0:1789302898.005138:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350560640 00000100:00100000:0.0:1789302898.005192:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 332 00000100:00100000:0.0:1789302898.005202:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+65:33482:x1876219350560640:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302898.005430:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:33482:x1876219350560640:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 229us (503us total) trans 4294968354 rc 0/0 00000100:00100000:0.0:1789302898.005437:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 332 00000100:00100000:3.0:1789302898.005662:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350560768 00000100:00100000:3.0:1789302898.005667:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350560768 00000100:00100000:3.0:1789302898.005705:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1581 00000100:00100000:3.0:1789302898.005713:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33489:x1876219350560768:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302898.005723:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.005743:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1284 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005791:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005807:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005819:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005833:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005841:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005847:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005891:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005899:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005907:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005913:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.005916:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005924:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005935:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.005938:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.005943:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005950:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518501bf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005962:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005969:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:1.0:1789302898.005975:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302898.005976:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005990:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.005993:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.005998:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.006005:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f0cdf2400) 00010000:00010000:3.0:1789302898.006009:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.006012:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.006019:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.006047:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501b8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1284 expref: 114 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.006057:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.006059:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.006063:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1284 expref: 114 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.006078:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3c00, rc 0) 00000100:00100000:3.0:1789302898.006116:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33489:x1876219350560768:12345-192.168.203.11@tcp:101:bash.0 Request processed in 404us (1146us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.006123:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1581 00000100:00100000:2.0:1789302898.011547:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.011780:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350560896 00000100:00100000:1.0:1789302898.011789:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350560896 00000100:00100000:1.0:1789302898.011842:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 81 00000100:00100000:1.0:1789302898.011851:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:2348:x1876219350560896:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:1.0:1789302898.011883:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.011902:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe4309518501cd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a128b expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.011965:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.011978:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.011989:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012003:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012010:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012021:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012078:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012085:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012092:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012098:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.012102:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012110:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012122:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.012125:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.012129:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012136:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe4309518501d4 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012148:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012154:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012161:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012166:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.012169:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012176:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012181:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x4b66ad81 (ffff960f2c94d900) 00010000:00010000:1.0:1789302898.012183:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.012186:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012191:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518501db lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012215:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe4309518501cd lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a128b expref: 114 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012225:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.012227:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302898.012230:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe4309518501cd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a128b expref: 114 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.012240:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8a00, rc -2) 00000100:00100000:1.0:1789302898.012608:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:2348:x1876219350560896:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 756us (1067us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302898.012620:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 81 00000100:00100000:1.0:1789302898.021248:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.021461:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350561152 00000100:00100000:3.0:1789302898.021471:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350561152 00000100:00100000:3.0:1789302898.021544:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1582 00000100:00100000:3.0:1789302898.021555:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:33483:x1876219350561152:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302898.021628:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.021646:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 66 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.021656:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.021689:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 66 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.021703:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 66 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.021729:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371183872:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302898.021801:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00af00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302898.026839:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44700 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371183872:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302898.026869:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.028774:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.028814:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.028882:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350561408 00000100:00100000:0.0:1789302898.028892:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350561408 00000100:00100000:0.0:1789302898.028953:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 82 00000100:00100000:0.0:1789302898.028964:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:2345:x1876219350561408:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302898.029005:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.029025:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518501e9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1299 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029078:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029093:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029105:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029119:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029125:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029132:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029183:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029191:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029198:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029205:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.029209:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029220:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029233:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.029236:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.029241:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029248:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518501f0 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029272:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029280:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029286:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029291:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.029294:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029300:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029308:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x36fdbbdd (ffff960f0cf83300) 00010000:00010000:0.0:1789302898.029311:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.029314:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029324:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518501f7 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x36fdbbdd bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029351:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518501e9 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1299 expref: 68 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029361:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.029363:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302898.029367:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518501e9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1299 expref: 68 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.029387:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b400, rc -2) 00000100:00100000:0.0:1789302898.029441:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:2345:x1876219350561408:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 480us (626us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302898.029449:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 82 00000100:00100000:3.0:1789302898.029565:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350561280 00000100:00100000:3.0:1789302898.029570:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350561280 00000100:00100000:3.0:1789302898.029606:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1583 00000100:00100000:3.0:1789302898.029615:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:33490:x1876219350561280:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302898.029624:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.029636:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1292 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029670:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029680:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029688:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029700:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029707:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029713:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029795:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029808:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029815:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029821:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.029823:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029830:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029839:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.029841:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.029844:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029850:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850205 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029861:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029867:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029874:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029880:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.029883:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029889:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029896:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f0cdf2400) 00010000:00010000:3.0:1789302898.029898:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.029901:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029906:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185020c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029929:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501fe lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1292 expref: 68 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029937:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.029940:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.029943:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3000/0x64fe4309518501fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1292 expref: 68 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.029953:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:3.0:1789302898.029989:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:33490:x1876219350561280:12345-192.168.203.11@tcp:101:rm.0 Request processed in 376us (1221us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.029996:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1583 00000100:00100000:0.0:1789302898.031795:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.031889:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350561536 00000100:00100000:2.0:1789302898.031897:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350561536 00010000:00010000:2.0:1789302898.031925:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 67 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.031960:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:2.0:1789302898.031968:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:2348:x1876219350561536:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302898.031980:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302898.031986:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 67 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.031998:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 67 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.032015:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe4309518501a3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a127d expref: 67 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.032027:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:2.0:1789302898.032032:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.032042:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.032049:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.032077:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302898.032122:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:2348:x1876219350561536:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 154us (334us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.032129:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 459 00010000:00010000:1.0:1789302898.032208:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032216:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032400:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032409:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032415:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032421:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.032426:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032434:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032442:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:1.0:1789302898.032445:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302898.032458:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.032469:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518501e2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.033479:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:33483:x1876219350561152:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 11922us (12236us total) trans 4294968355 rc 0/0 00000100:00100000:2.0:1789302898.033489:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1582 00000100:00100000:0.0:1789302898.034030:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.034085:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350561664 00000100:00100000:2.0:1789302898.034101:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350561664 00000100:00100000:2.0:1789302898.034140:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1584 00000100:00100000:2.0:1789302898.034147:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:33490:x1876219350561664:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302898.034162:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.034178:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe430951850213 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12a0 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034234:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034246:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034257:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034269:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034276:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034282:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034326:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034333:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034339:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034345:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.034349:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034355:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034365:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.034368:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.034383:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034389:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185021a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034400:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034406:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034413:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034417:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.034419:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034425:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034437:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f09ae1f00) 00010000:00010000:2.0:1789302898.034440:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.034442:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034457:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850221 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034483:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe430951850213 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12a0 expref: 67 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034492:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.034495:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.034499:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6400/0x64fe430951850213 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12a0 expref: 67 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.034531:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:2.0:1789302898.034565:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+66:33490:x1876219350561664:12345-192.168.203.11@tcp:101:rm.0 Request processed in 418us (537us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.034572:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1584 00000040:00080000:1.0:1789302898.036129:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302898.036135:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302898.036180:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 50 count 8 00000040:00080000:1.0:1789302898.036187:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302898.036660:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffa7b80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371184000:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302898.036731:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f123a6000 going to sleep for 11 seconds 00000100:00100000:1.0:1789302898.048926:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffa7b80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371184000:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302898.056287:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.056685:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350561792 00000100:00100000:1.0:1789302898.056843:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.056855:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350561792 00000100:00100000:1.0:1789302898.056892:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.056998:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350562048 00000100:00100000:2.0:1789302898.057003:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350562048 00000100:00100000:2.0:1789302898.057038:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 333 00000100:00100000:2.0:1789302898.057048:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:33483:x1876219350562048:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302898.057302:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+68:33483:x1876219350562048:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 254us (459us total) trans 4294968356 rc 0/0 00000100:00100000:2.0:1789302898.057314:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 333 00000100:00100000:3.0:1789302898.076133:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350562176 00000100:00100000:3.0:1789302898.076150:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350562176 00000100:00100000:3.0:1789302898.077115:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1585 00000100:00100000:3.0:1789302898.077126:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+68:33492:x1876219350561792:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:3.0:1789302898.077159:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302898.077274:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1586 00000100:00100000:3.0:1789302898.077284:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+68:33492:x1876219350561792:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 159us (21004us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.077290:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+68:33491:x1876219350562176:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000100:00100000:3.0:1789302898.077298:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1585 00000004:00000080:1.0:1789302898.077310:0:5844:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x31:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302898.077361:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:33491:x1876219350562176:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 77us (20468us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.077368:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1586 00000100:00100000:1.0:1789302898.086279:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.089168:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350562304 00000100:00100000:3.0:1789302898.089185:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350562304 00000100:00100000:2.0:1789302898.089369:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1587 00000100:00100000:2.0:1789302898.089393:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:33493:x1876219350562304:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.089413:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.089437:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850228 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12a7 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089519:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089534:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089545:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089560:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089567:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089574:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089630:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089638:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089649:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089655:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.089659:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089667:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089679:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.089682:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.089688:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089695:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185022f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089707:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089714:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089719:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089724:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.089727:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089737:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089744:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f09ae1f00) 00010000:00010000:2.0:1789302898.089746:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.089748:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089753:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850236 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089977:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850228 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12a7 expref: 68 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.089988:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.089990:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.089995:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850228 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12a7 expref: 68 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.090006:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:2.0:1789302898.090071:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef32d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+67:33493:x1876219350562304:12345-192.168.203.11@tcp:101:cp.0 Request processed in 681us (3801us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.090079:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1587 00000100:00100000:0.0:1789302898.092591:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.092660:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.092698:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350562560 00000100:00100000:0.0:1789302898.092709:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350562560 00000100:00100000:1.0:1789302898.092742:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350562944 00000100:00100000:1.0:1789302898.092749:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350562944 00000100:00100000:0.0:1789302898.092773:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1588 00000100:00100000:1.0:1789302898.092782:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1589 00000100:00100000:0.0:1789302898.092788:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+68:33492:x1876219350562560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00000100:00100000:1.0:1789302898.092797:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+68:33493:x1876219350562944:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302898.092816:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.092817:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.092838:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850244 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12ae expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.092839:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095185023d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12bc expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.092910:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.092922:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.092933:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.092954:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850252 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.092960:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.092970:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850252 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.092972:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.092990:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850252 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.092991:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302898.093011:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x32:0x0]: returning size 158 00010000:00010000:1.0:1789302898.093063:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093071:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093077:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093083:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.093087:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093098:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093110:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.093113:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.093118:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093123:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe43095185024b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093135:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.093140:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850244 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ae expref: 72 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093143:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093160:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093166:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.093169:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.093170:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850252 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ae expref: 71 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093183:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.093188:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:0.0:1789302898.093194:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302898.093196:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f2c94d900) 00010000:00010000:0.0:1789302898.093197:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00010000:00010000:1.0:1789302898.093201:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.093206:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093216:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850259 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093242:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095185023d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12bc expref: 71 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093251:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.093254:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.093258:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095185023d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12bc expref: 71 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.093265:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8a00, rc 0) 00000100:00100000:1.0:1789302898.093287:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+70:33493:x1876219350562944:12345-192.168.203.11@tcp:101:cp.0 Request processed in 495us (629us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302898.093293:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1589 00000100:00100000:0.0:1789302898.093422:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+69:33492:x1876219350562560:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 476us (678us total) trans 4294968357 rc 0/0 00000100:00100000:0.0:1789302898.093430:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1588 00000100:00100000:2.0:1789302898.096423:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.099768:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350562816 00000100:00100000:0.0:1789302898.099784:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350562816 00000100:00100000:0.0:1789302898.099851:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1590 00000100:00100000:0.0:1789302898.099862:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+69:33491:x1876219350562816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302898.099882:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.099903:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850267 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12b5 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.100016:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185026e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.100031:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185026e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.100043:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185026e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302898.100053:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x31:0x0]: returning size 158 00010000:00010000:0.0:1789302898.100181:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850267 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12b5 expref: 72 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.100201:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185026e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12b5 expref: 71 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.100209:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:0.0:1789302898.100213:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302898.100216:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc 0) 00000100:00100000:1.0:1789302898.101102:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:33491:x1876219350562816:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1237us (4686us total) trans 4294968358 rc 0/0 00000100:00100000:1.0:1789302898.101111:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1590 00000100:00100000:3.0:1789302898.102185:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.102262:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350563200 00000100:00100000:1.0:1789302898.102272:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350563200 00000100:00100000:1.0:1789302898.102322:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1591 00000100:00100000:1.0:1789302898.102332:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:33493:x1876219350563200:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302898.102363:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.102389:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095185027c lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12c3 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.102561:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.102577:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 112 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.102585:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.102616:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 112 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.102630:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 112 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.102642:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371184128:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302898.102711:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f123a63c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302898.108612:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.109109:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.109141:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.109192:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350563328 00000100:00100000:0.0:1789302898.109204:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350563328 00000100:00100000:0.0:1789302898.109275:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 74 00000100:00100000:0.0:1789302898.109285:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+73:2345:x1876219350563328:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302898.109300:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302898.109324:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850252 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ae expref: 73 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.109363:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:0.0:1789302898.109369:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302898.109435:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:2345:x1876219350563328:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 151us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.109443:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 74 00000100:00100000:0.0:1789302898.110694:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350563456 00000100:00100000:0.0:1789302898.110703:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350563456 00000100:00100000:0.0:1789302898.110746:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350563584 00000100:00100000:0.0:1789302898.110750:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350563584 00000100:00100000:0.0:1789302898.110780:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 453 00000100:00100000:0.0:1789302898.110789:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2348:x1876219350563456:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302898.110832:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.110851:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.110859:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x20:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789302898.110863:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302898.110868:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 4/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.110875:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.110894:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:0.0:1789302898.110900:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.110912:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 16 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.110927:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371184256:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302898.111604:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 454 00000100:00100000:1.0:1789302898.111615:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350563584:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302898.111640:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111653:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 13 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111660:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x20:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302898.111664:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302898.111667:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111689:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 13 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111700:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 13 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.111710:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371184384:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302898.111789:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f123a6000 going to sleep for 11 seconds 00000100:00080000:1.0:1789302898.111809:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302898.111821:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350563712 00000100:00100000:1.0:1789302898.111826:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350563712 00000100:00100000:1.0:1789302898.111862:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350563840 00000100:00100000:1.0:1789302898.111865:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350563840 00010000:00010000:1.0:1789302898.111881:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 113 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.111901:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 460 00000100:00100000:1.0:1789302898.111907:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:2348:x1876219350563840:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302898.111918:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302898.111924:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 5/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 113 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111935:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 113 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111966:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.111970:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111979:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.111986:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT flags: 0x50290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112002:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.112026:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:2348:x1876219350563840:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 119us (1131us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.112034:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 460 00000100:00100000:1.0:1789302898.112050:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 75 00000100:00100000:1.0:1789302898.112056:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:2348:x1876219350563712:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302898.112064:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302898.112076:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185026e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12b5 expref: 72 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112090:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:1.0:1789302898.112093:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.112112:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:2348:x1876219350563712:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 56us (1237us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.112117:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 75 00010000:00010000:1.0:1789302898.112167:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9400/0x64fe43095185016b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1268 expref: 112 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.112179:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371184128:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302898.112197:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112206:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112213:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112231:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112239:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112245:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.112760:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2f00 going to sleep for 11 seconds 00000004:00080000:1.0:1789302898.112821:0:18427:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x21:0x0], index 0: 0 00010000:00010000:1.0:1789302898.112974:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112981:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.112986:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113009:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113014:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113018:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113021:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.113025:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113029:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113034:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.113036:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.113039:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113043:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe430951850283 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302898.113049:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000f163ff03 reply state 00000000752c36b0 transno 4294968359 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.113053:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302898.113075:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe43095185027c lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 72 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113084:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 71 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.113093:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:1.0:1789302898.113095:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302898.113096:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8800, rc 0) 00000100:00100000:1.0:1789302898.113116:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:33493:x1876219350563200:12345-192.168.203.11@tcp:101:cp.0 Request processed in 10788us (10937us total) trans 4294968359 rc 0/0 00000100:00100000:1.0:1789302898.113121:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1591 00010000:00010000:0.0:1789302898.124073:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.124090:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.124097:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.124104:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.124109:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.124116:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.124132:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f2c94d900) 00010000:00010000:0.0:1789302898.124135:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.124140:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.124147:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850298 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.127742:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.127777:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.127802:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.127875:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350564096 00000100:00100000:2.0:1789302898.127884:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350564096 00000100:00100000:2.0:1789302898.127937:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 455 00000100:00100000:2.0:1789302898.127947:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2348:x1876219350564096:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.127965:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.127984:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12ca expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302898.128009:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x21:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302898.128028:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789302898.128281:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.128291:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 15 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.128312:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 15 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.128324:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7c00, rc 0) 00000100:00100000:2.0:1789302898.128390:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2348:x1876219350564096:12345-192.168.203.11@tcp:101:cp.0 Request processed in 444us (587us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.128398:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 455 00000100:00100000:2.0:1789302898.128759:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371184256:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302898.128784:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.129696:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371184384:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302898.129716:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.131595:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350563968 00000100:00100000:0.0:1789302898.131606:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350563968 00002000:00100000:0.0:1789302898.131651:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x20:0x0] (0->157) req@ffff960f3ef33100 x1876219350563968/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 648/0 e 0 to 0 dl 1789302909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302898.131673:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef33100 with x1876219350563968 ext(0->157) 00010000:00010000:0.0:1789302898.131681:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 15 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.131695:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 15 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302898.131703:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3ef33100 00000100:00100000:0.0:1789302898.131827:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350564224 00000100:00100000:0.0:1789302898.131831:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350564224 00002000:00100000:0.0:1789302898.131850:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_002: refresh rw locks for [0x280000400:0x20:0x0] (0->157) req@ffff960f3f756a00 x1876219350564224/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 648/0 e 0 to 0 dl 1789302909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302898.131859:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f756a00 with x1876219350564224 ext(0->157) 00010000:00010000:0.0:1789302898.131864:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 17 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.131875:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 17 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302898.131882:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3f756a00 00000100:00100000:1.0:1789302898.131952:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 16 00000100:00100000:1.0:1789302898.131964:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2346:x1876219350563968:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302898.132670:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2346:x1876219350563968:12345-192.168.203.11@tcp:4:cp.0 Request processed in 707us (4934us total) trans 4294967386 rc 0/0 00000100:00100000:1.0:1789302898.132679:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 16 00002000:00100000:1.0:1789302898.132686:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x20:0x0] (0->157) req@ffff960f3ef33100 x1876219350563968/t4294967386(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 648/448 e 0 to 0 dl 1789302909 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302898.132695:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef33100 with x1876219350563968 ext(0->157) 00010000:00010000:1.0:1789302898.132701:0:18429:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 17 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.132712:0:18429:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 17 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302898.132719:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3ef33100 00000100:00100000:1.0:1789302898.132740:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 17 00000100:00100000:1.0:1789302898.132748:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2345:x1876219350564224:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:3.0:1789302898.133551:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.135208:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2345:x1876219350564224:12345-192.168.203.11@tcp:4:cp.0 Request processed in 2454us (7422us total) trans 4294967384 rc 0/0 00000100:00100000:1.0:1789302898.135219:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 17 00002000:00100000:1.0:1789302898.135230:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_004: refresh rw locks for [0x280000400:0x20:0x0] (0->157) req@ffff960f3f756a00 x1876219350564224/t4294967384(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 648/448 e 0 to 0 dl 1789302909 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302898.135240:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f756a00 with x1876219350564224 ext(0->157) 00010000:00010000:1.0:1789302898.135249:0:18429:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 19 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.135274:0:18429:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 19 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302898.135283:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3f756a00 00000100:00100000:1.0:1789302898.135316:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350564352 00000100:00100000:1.0:1789302898.135323:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350564352 00000100:00100000:1.0:1789302898.135366:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:1.0:1789302898.135386:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:33493:x1876219350564352:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302898.135398:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.135414:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12d1 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.135439:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.135469:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.135474:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.135481:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 18 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.135540:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.135553:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 18 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.135566:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f754000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371184512:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302898.135616:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0c757cc0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302898.138007:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f754000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371184512:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302898.138061:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 3/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 18 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.138081:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9e00, rc 0) 00000100:00100000:1.0:1789302898.138134:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:33493:x1876219350564352:12345-192.168.203.11@tcp:101:cp.0 Request processed in 2760us (4585us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.138142:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 456 00000100:00100000:3.0:1789302898.140595:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.140762:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350564480 00000100:00100000:1.0:1789302898.140771:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350564480 00000100:00100000:3.0:1789302898.141922:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.142055:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350564608 00000100:00100000:1.0:1789302898.142062:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350564608 00010000:00010000:1.0:1789302898.142100:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 18 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.142148:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 461 00000100:00100000:1.0:1789302898.142158:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd05850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350564608:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302898.142172:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302898.142195:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 18 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.142208:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 18 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.142230:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e2654d75/0x64fe43095184ff9d lrc: 0/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a111f expref: 18 pid: 18578 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.142249:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302898.142320:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 19 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.142355:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 462 00000100:00100000:2.0:1789302898.142386:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350564480:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302898.142396:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302898.142411:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 19 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.142414:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302898.142426:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.142428:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 19 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.142453:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a20dbdc7/0x64fe4309518500c3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11f1 expref: 19 pid: 6697 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.142464:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302898.142467:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(1 0) 00010000:00010000:2.0:1789302898.142471:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.142481:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.142541:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302898.142574:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.142586:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.142591:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350564480:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 227us (2001us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.142607:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 462 00010000:00010000:0.0:1789302898.142634:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.142645:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.142654:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.142661:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302898.142666:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.142676:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.142690:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.142699:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000078d1b910/0x64fe43095185028a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.142764:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350563456:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 31976us (34161us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.142776:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 453 00010000:00010000:1.0:1789302898.142999:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.143076:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.143115:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd05850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350564608:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 959us (1196us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.143126:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 461 00010000:00010000:3.0:1789302898.144098:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.144110:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.144126:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.144231:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350564736 00000100:00100000:0.0:1789302898.144239:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350564736 00010000:00010000:3.0:1789302898.144258:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.144266:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 17 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.144268:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.144277:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.144284:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302898.144288:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.144296:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.144300:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:0.0:1789302898.144308:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350564736:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302898.144311:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.144318:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302898.144319:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850291 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.144331:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 17 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.144341:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 17 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.144355:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e9bed7d7/0x64fe4309518502ad lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ca expref: 17 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.144381:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.144386:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00000100:00100000:3.0:1789302898.144394:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350563584:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 32782us (35252us total) trans 0 rc 0/0 00010000:00010000:0.0:1789302898.144394:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 16 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.144405:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 16 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.144406:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 454 00010000:00010000:0.0:1789302898.144447:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 15 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.144573:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371184640:192.168.203.11@tcp:105: 00000100:00100000:0.0:1789302898.144620:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2780 going to sleep for 11 seconds 00000100:00100000:3.0:1789302898.145197:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.145247:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350564992 00000100:00100000:0.0:1789302898.145253:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350564992 00000100:00100000:0.0:1789302898.145298:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1592 00000100:00100000:0.0:1789302898.145306:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:33492:x1876219350564992:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302898.145692:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:33492:x1876219350564992:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 386us (496us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.145700:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1592 00000100:00100000:3.0:1789302898.149785:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.149824:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.149918:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350565248 00000100:00100000:1.0:1789302898.149929:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350565248 00000100:00100000:1.0:1789302898.149992:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1593 00000100:00100000:1.0:1789302898.150002:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:33491:x1876219350565248:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302898.150100:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:33491:x1876219350565248:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 98us (322us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.150107:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1593 00000100:00100000:0.0:1789302898.150467:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267df80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371184640:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302898.150469:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350565504 00000100:00100000:1.0:1789302898.150477:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350565504 00010000:00010000:0.0:1789302898.150494:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.150537:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 334 00000100:00100000:0.0:1789302898.150546:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2348:x1876219350564736:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 6237us (6422us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.150551:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+71:33493:x1876219350565504:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302898.150563:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:1.0:1789302898.151023:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:33493:x1876219350565504:12345-192.168.203.11@tcp:35:cp.0 Request processed in 477us (1197us total) trans 4294968360 rc 0/0 00000100:00100000:1.0:1789302898.151031:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 334 00000100:00100000:2.0:1789302898.151163:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.151473:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350565632 00000100:00100000:1.0:1789302898.151478:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350565632 00000100:00100000:1.0:1789302898.151533:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:1.0:1789302898.151541:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350565632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302898.151552:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.151571:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12d8 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302898.151595:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x20:0x0] initial LVB size: 158, mtime: 0x6aa69871, atime: 0x0, ctime: 0x6aa69871, blocks: 0x1 00010000:00010000:1.0:1789302898.151610:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789302898.151616:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.151624:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.151639:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.151650:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9400, rc 0) 00000100:00100000:1.0:1789302898.151689:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350565632:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 150us (527us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.151695:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 457 00000100:00100000:2.0:1789302898.155040:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.155325:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350565888 00000100:00100000:1.0:1789302898.155334:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350565888 00000100:00100000:1.0:1789302898.155430:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1594 00000100:00100000:1.0:1789302898.155441:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:33492:x1876219350565888:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302898.155571:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:33492:x1876219350565888:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 132us (538us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.155579:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1594 00000100:00100000:3.0:1789302898.155991:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.156050:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350566016 00000100:00100000:2.0:1789302898.156056:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350566016 00000100:00100000:2.0:1789302898.156097:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:2.0:1789302898.156104:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2347:x1876219350566016:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302898.156115:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.156135:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12df expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302898.156160:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x20:0x0] initial LVB size: 158, mtime: 0x6aa69871, atime: 0x0, ctime: 0x6aa69871, blocks: 0x1 00010000:00010000:2.0:1789302898.156176:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789302898.156180:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.156194:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 16 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.156210:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 16 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.156222:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6400, rc 0) 00000100:00100000:2.0:1789302898.156262:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350566016:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 154us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.156269:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 458 00000100:00100000:0.0:1789302898.160425:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.160686:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.160901:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350566144 00000100:00100000:2.0:1789302898.160911:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350566144 00002000:00100000:2.0:1789302898.160962:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x20:0x0] (0->4095) req@ffff960e0267df80 x1876219350566144/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 488/0 e 0 to 0 dl 1789302909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302898.160974:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0267df80 with x1876219350566144 ext(0->4095) 00002000:00010000:2.0:1789302898.160980:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0267df80 00000100:00100000:2.0:1789302898.161009:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 132 00000100:00100000:0.0:1789302898.161103:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350566400 00000100:00100000:0.0:1789302898.161108:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350566400 00000100:00100000:0.0:1789302898.161154:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1595 00000100:00100000:0.0:1789302898.161163:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:33491:x1876219350566400:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:0.0:1789302898.161264:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+72:33491:x1876219350566400:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 101us (578us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.161273:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1595 00000100:00100000:2.0:1789302898.161303:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2347:x1876219350566144:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:2.0:1789302898.161475:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267df80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2347:x1876219350566144:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 173us (1060us total) trans 0 rc 4096/4096 00000100:00100000:2.0:1789302898.161482:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 132 00002000:00100000:2.0:1789302898.161491:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_001: refresh rw locks for [0x280000400:0x20:0x0] (0->4095) req@ffff960e0267df80 x1876219350566144/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 488/4536 e 0 to 0 dl 1789302909 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302898.161525:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0267df80 with x1876219350566144 ext(0->4095) 00002000:00010000:2.0:1789302898.161529:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960e0267df80 00000100:00100000:3.0:1789302898.168280:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.168409:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350566528 00000100:00100000:0.0:1789302898.168420:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350566528 00002000:00100000:0.0:1789302898.168644:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x20:0x0] (0->4095) req@ffff960f3f7c9180 x1876219350566528/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 488/0 e 0 to 0 dl 1789302909 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302898.168657:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7c9180 with x1876219350566528 ext(0->4095) 00002000:00010000:0.0:1789302898.168664:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f7c9180 00000100:00100000:3.0:1789302898.173214:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 133 00000100:00100000:3.0:1789302898.173235:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350566528:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302898.173450:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350566528:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 222us (5177us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302898.173459:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 133 00002000:00100000:3.0:1789302898.173470:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_000: refresh rw locks for [0x240000400:0x20:0x0] (0->4095) req@ffff960f3f7c9180 x1876219350566528/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:474/0 lens 488/4536 e 0 to 0 dl 1789302909 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302898.173479:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f7c9180 with x1876219350566528 ext(0->4095) 00002000:00010000:3.0:1789302898.173485:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f7c9180 00000100:00100000:1.0:1789302898.173884:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.178595:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.178688:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350566656 00000100:00100000:1.0:1789302898.178697:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350566656 00000100:00100000:1.0:1789302898.178744:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350566784 00000100:00100000:1.0:1789302898.178746:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350566784 00000100:00100000:1.0:1789302898.178761:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1596 00000100:00100000:1.0:1789302898.178769:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+73:33492:x1876219350566656:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302898.178790:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.178811:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518502c9 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12e6 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.178856:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.178869:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.178881:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.178975:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518502c9 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.178993:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 74 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.179000:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:1.0:1789302898.179003:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302898.179007:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302898.179055:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+74:33492:x1876219350566656:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 286us (5172us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.179062:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1596 00000100:00100000:1.0:1789302898.179077:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1597 00000100:00100000:1.0:1789302898.179083:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+73:33491:x1876219350566784:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302898.179090:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.179097:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518502d7 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12ed expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.179117:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 3/1,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.179125:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.179131:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.179178:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518502d7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 75 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.179189:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 74 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.179195:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:1.0:1789302898.179198:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302898.179200:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9600, rc 0) 00000100:00100000:1.0:1789302898.179217:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+74:33491:x1876219350566784:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 135us (631us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.179222:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1597 00000100:00100000:1.0:1789302898.189176:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.189324:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.189417:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350566912 00000100:00100000:2.0:1789302898.189427:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350566912 00000100:00100000:3.0:1789302898.189435:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350567040 00000100:00100000:3.0:1789302898.189442:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350567040 00000100:00100000:2.0:1789302898.189581:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1598 00000100:00100000:2.0:1789302898.189592:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+75:33491:x1876219350566912:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302898.189621:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x31:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302898.189713:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+75:33491:x1876219350566912:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 123us (545us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.189720:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1598 00000100:00100000:2.0:1789302898.189739:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1599 00000100:00100000:2.0:1789302898.189745:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+74:33492:x1876219350567040:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302898.189757:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x32:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302898.189805:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+74:33492:x1876219350567040:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 59us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.189813:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1599 00000100:00100000:1.0:1789302898.201768:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.201816:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.201958:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350567168 00000100:00100000:0.0:1789302898.201968:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350567168 00000100:00100000:0.0:1789302898.202025:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350567296 00000100:00100000:0.0:1789302898.202039:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350567296 00000100:00100000:0.0:1789302898.202070:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:0.0:1789302898.202079:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2346:x1876219350567168:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302898.202121:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 3/1,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202137:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.202142:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.202147:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202162:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202194:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202202:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202211:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202218:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302898.202222:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202230:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202244:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202251:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000005fc3651e/0x64fe4309518502e5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.202318:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2346:x1876219350567168:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 239us (559us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.202326:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 459 00000100:00100000:0.0:1789302898.202343:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 460 00000100:00100000:0.0:1789302898.202351:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350567296:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302898.202387:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 3/1,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202395:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.202398:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.202402:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202412:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202430:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202437:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202589:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202596:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302898.202599:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202606:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202621:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.202627:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518502ec lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.202661:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350567296:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 310us (845us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.202668:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 460 00000100:00100000:0.0:1789302898.203703:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.203785:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350567424 00000100:00100000:2.0:1789302898.203793:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350567424 00000100:00100000:2.0:1789302898.203838:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1600 00000100:00100000:2.0:1789302898.203847:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+112:33497:x1876219350567424:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302898.203862:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.203877:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518502f3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12f4 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.203934:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.203947:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.203957:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.203971:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.203977:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.203984:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204185:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe430951850308 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204194:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe430951850308 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204202:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe430951850308 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302898.204232:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 158 00000004:00200000:2.0:1789302898.204237:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302898.204251:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe430951850308 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204261:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe430951850308 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204269:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204276:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204282:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204288:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.204292:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204299:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204308:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.204312:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.204316:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204321:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518502fa lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204332:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204338:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204343:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204348:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.204351:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204356:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204361:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f10d3c800) 00010000:00010000:2.0:1789302898.204364:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.204366:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204382:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850301 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204412:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518502f3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12f4 expref: 114 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204426:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe430951850308 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12f4 expref: 113 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.204434:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:2.0:1789302898.204436:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302898.204439:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6800, rc 0) 00000100:00100000:2.0:1789302898.204757:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33497:x1876219350567424:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 909us (1055us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.204765:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1600 00000100:00100000:1.0:1789302898.207828:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.208537:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.208615:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350567808 00000100:00100000:2.0:1789302898.208615:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350567680 00000100:00100000:3.0:1789302898.208623:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350567808 00000100:00100000:2.0:1789302898.208623:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350567680 00000100:00100000:3.0:1789302898.208674:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 461 00000100:00100000:3.0:1789302898.208683:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350567808:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302898.208698:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:2.0:1789302898.208698:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1601 00000100:00100000:2.0:1789302898.208705:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+74:33491:x1876219350567680:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302898.208714:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c810bd51/0x64fe43095185030f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a12fb expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.208756:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.208767:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371184768:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302898.208771:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 74 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.208846:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00aa80 going to sleep for 11 seconds 00010000:00010000:2.0:1789302898.208846:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.208869:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 74 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.208881:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 74 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.208893:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371184896:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302898.208918:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b0c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302898.210309:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.210397:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350568064 00000100:00100000:3.0:1789302898.210405:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350568064 00000100:00100000:1.0:1789302898.210637:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1602 00000100:00100000:1.0:1789302898.210648:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+75:33492:x1876219350568064:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:1.0:1789302898.210699:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.210712:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.210719:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 4/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.210742:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.210752:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.210762:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f756300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371185024:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302898.210810:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0c76f9c0 going to sleep for 11 seconds 00000100:00100000:0.0:1789302898.212131:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371184896:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302898.212155:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302898.212208:0:13439:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1789302898.212212:0:13439:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb mtime: 0 -> 1789302897 00002000:00010000:3.0:1789302898.212214:0:13439:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb atime: 0 -> 1789302897 00002000:00010000:3.0:1789302898.212216:0:13439:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x21:0x0] updating lvb ctime: 0 -> 1789302897 00000100:00100000:3.0:1789302898.212232:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7caa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371184768:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302898.212255:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000c810bd51/0x64fe43095185030f lrc: 1/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12fb expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.212274:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000c810bd51/0x64fe43095185030f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12fb expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.212287:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:3.0:1789302898.213226:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350567808:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4540us (4685us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.213236:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 461 00000100:00100000:0.0:1789302898.213322:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.213660:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350568192 00000100:00100000:2.0:1789302898.213669:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350568192 00010000:00010000:2.0:1789302898.213698:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 76 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.213731:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00100000:2.0:1789302898.213739:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:2348:x1876219350568192:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302898.213753:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302898.213761:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 76 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.213772:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 76 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.213792:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518502de lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12ed expref: 76 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.213808:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:2.0:1789302898.213813:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.213822:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.213835:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.214018:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302898.214060:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+75:2348:x1876219350568192:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 322us (738us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.214068:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00100000:1.0:1789302898.214237:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f756300 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371185024:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302898.214255:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214388:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214394:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214589:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214597:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214603:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214606:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.214609:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214614:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214621:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:3.0:1789302898.214623:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302898.214625:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.214629:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850316 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.214718:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+75:33491:x1876219350567680:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 6012us (6897us total) trans 4294968361 rc 0/0 00000100:00100000:3.0:1789302898.214726:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1601 00000100:00100000:0.0:1789302898.216658:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.216745:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350568320 00000100:00100000:1.0:1789302898.216754:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350568320 00010000:00010000:1.0:1789302898.216788:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.216869:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 465 00000100:00100000:1.0:1789302898.216878:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+75:2348:x1876219350568320:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302898.216893:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302898.216899:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.216910:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.216929:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518502d0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12e6 expref: 75 pid: 5844 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.216954:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:1.0:1789302898.216959:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.216968:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.216974:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.217002:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.217050:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+74:2348:x1876219350568320:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 172us (395us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.217058:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 465 00010000:00010000:2.0:1789302898.217106:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217113:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217285:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217293:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217300:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217305:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.217310:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217317:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217326:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:2.0:1789302898.217329:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302898.217333:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 2/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.217337:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185031d lrc: 0/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.217463:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+74:33492:x1876219350568064:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 6817us (7158us total) trans 4294968362 rc 0/0 00000100:00100000:2.0:1789302898.217471:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1602 00000040:00080000:0.0:1789302898.217542:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302898.217547:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302898.217927:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 8 00000040:00080000:0.0:1789302898.217942:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302898.220863:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.220903:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.220913:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350568448 00000100:00100000:2.0:1789302898.220923:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350568448 00000100:00100000:0.0:1789302898.220934:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.220976:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1603 00000100:00100000:2.0:1789302898.220986:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33497:x1876219350568448:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302898.221005:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:3.0:1789302898.221015:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350568704 00000100:00100000:3.0:1789302898.221021:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350568704 00010000:00010000:2.0:1789302898.221023:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850324 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1302 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.221059:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350568960 00000100:00100000:3.0:1789302898.221063:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350568960 00000100:00100000:3.0:1789302898.221078:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:3.0:1789302898.221086:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22333450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+75:33491:x1876219350568704:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00010000:00010000:2.0:1789302898.221095:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.221108:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.221118:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302898.221142:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:2.0:1789302898.221146:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302898.221167:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.221184:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 336 00010000:00010000:2.0:1789302898.221191:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850324 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 115 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.221199:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33492:x1876219350568960:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00010000:00010000:2.0:1789302898.221215:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 114 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.221223:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.221226:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302898.221230:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc 0) 00000100:00100000:2.0:1789302898.221544:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33497:x1876219350568448:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 558us (687us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302898.221547:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22333450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+77:33491:x1876219350568704:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 458us (640us total) trans 4294968363 rc 0/0 00000100:00100000:2.0:1789302898.221554:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1603 00000100:00100000:3.0:1789302898.221562:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 335 00000100:00100000:1.0:1789302898.221576:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33492:x1876219350568960:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 379us (638us total) trans 4294968364 rc 0/0 00000100:00100000:1.0:1789302898.221585:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 336 00000100:00100000:3.0:1789302898.223470:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.223557:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350569088 00000100:00100000:0.0:1789302898.223565:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350569088 00000100:00100000:0.0:1789302898.223614:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 462 00000100:00100000:0.0:1789302898.223625:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350569088:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302898.223640:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.223655:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e588a72d/0x64fe430951850332 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1309 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.223707:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371185152:192.168.203.11@tcp:106: 00000100:00100000:0.0:1789302898.223780:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302898.225741:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371185152:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302898.225775:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000e588a72d/0x64fe430951850332 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1309 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.225797:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e588a72d/0x64fe430951850332 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1309 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.225811:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc 0) 00000100:00100000:2.0:1789302898.225859:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350569088:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2236us (2393us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.225868:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 462 00000100:00100000:3.0:1789302898.229497:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.229621:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350569344 00000100:00100000:2.0:1789302898.229630:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350569344 00000100:00100000:2.0:1789302898.229698:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1604 00000100:00100000:2.0:1789302898.229708:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33497:x1876219350569344:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:2.0:1789302898.230726:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].50, 136 off14992 00000100:00100000:2.0:1789302898.230893:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33497:x1876219350569344:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 1184us (1447us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.230903:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1604 00000100:00100000:3.0:1789302898.243631:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.243909:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350569472 00000100:00100000:2.0:1789302898.243919:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350569472 00000100:00100000:2.0:1789302898.243985:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1605 00000100:00100000:2.0:1789302898.243996:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33499:x1876219350569472:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:2.0:1789302898.244016:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.244037:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850339 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1310 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244103:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244117:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244126:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244137:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244142:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244146:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244189:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244194:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244198:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244201:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.244204:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244209:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244217:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.244219:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.244223:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244226:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850340 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244234:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244238:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244242:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244246:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.244248:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244251:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244255:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f10d3c800) 00010000:00010000:2.0:1789302898.244441:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.244450:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244456:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850347 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244489:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850339 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1310 expref: 115 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244499:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.244608:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.244613:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850339 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1310 expref: 115 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.244622:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:2.0:1789302898.244723:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33499:x1876219350569472:12345-192.168.203.11@tcp:101:bash.0 Request processed in 721us (1093us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.244731:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1605 00000100:00100000:0.0:1789302898.251826:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.252446:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350569600 00000100:00100000:2.0:1789302898.252456:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350569600 00000100:00100000:2.0:1789302898.252559:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1606 00000100:00100000:2.0:1789302898.252570:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33500:x1876219350569600:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302898.252595:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.252615:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095185034e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1317 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252701:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252716:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252727:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252742:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252750:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252756:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252809:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252817:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252823:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252830:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.252834:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252841:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252853:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.252856:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.252861:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252866:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850355 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252879:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252884:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252890:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252895:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.252898:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252903:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252913:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f10d3c800) 00010000:00010000:2.0:1789302898.252916:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.252918:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252924:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185035c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252955:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095185034e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1317 expref: 77 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252965:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.252967:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.252970:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe43095185034e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1317 expref: 77 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.252979:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:2.0:1789302898.253031:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33500:x1876219350569600:12345-192.168.203.11@tcp:101:rm.0 Request processed in 464us (1210us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.253038:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1606 00000100:00100000:0.0:1789302898.258098:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.258245:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350569728 00000100:00100000:2.0:1789302898.258255:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350569728 00000100:00100000:2.0:1789302898.258363:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1607 00000100:00100000:2.0:1789302898.258389:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33500:x1876219350569728:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302898.258408:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.258451:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850363 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a131e expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258549:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258564:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258576:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258591:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258597:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258604:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258656:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258663:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258669:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258675:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.258680:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258690:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258714:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.258717:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.258722:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258728:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe43095185036a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258741:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258746:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258752:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258757:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.258759:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258764:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258771:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f10d3c500) 00010000:00010000:2.0:1789302898.258773:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.258776:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258785:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850371 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258815:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850363 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a131e expref: 77 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258824:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.258826:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.258829:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850363 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a131e expref: 77 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.258839:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516200, rc 0) 00000100:00100000:2.0:1789302898.258903:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33500:x1876219350569728:12345-192.168.203.11@tcp:101:rm.0 Request processed in 530us (814us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.258910:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1607 00000100:00100000:0.0:1789302898.269358:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.269473:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350569856 00000100:00100000:2.0:1789302898.269483:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350569856 00000100:00100000:2.0:1789302898.269560:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1608 00000100:00100000:2.0:1789302898.269570:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33501:x1876219350569856:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.269588:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.269608:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850378 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1325 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269670:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269684:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269705:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269728:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269737:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269743:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269796:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269804:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269811:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269817:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.269821:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269829:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.269842:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.269844:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.270006:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270015:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185037f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270031:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270038:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270043:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270048:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517e00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.270051:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270058:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270065:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f10d3c500) 00010000:00010000:2.0:1789302898.270068:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.270071:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270077:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850386 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270109:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850378 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1325 expref: 77 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270125:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.270127:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.270130:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850378 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1325 expref: 77 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.270139:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516200, rc 0) 00000100:00100000:2.0:1789302898.270290:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33501:x1876219350569856:12345-192.168.203.11@tcp:101:cp.0 Request processed in 720us (940us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.270298:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1608 00000100:00100000:3.0:1789302898.273845:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.273953:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350569984 00000100:00100000:0.0:1789302898.273963:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350569984 00000100:00100000:0.0:1789302898.274017:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1609 00000100:00100000:0.0:1789302898.274036:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33501:x1876219350569984:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.274055:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.274076:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185038d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a132c expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274132:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274145:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274156:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274174:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274181:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274188:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274240:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274248:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274255:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274274:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.274279:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274287:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274299:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.274302:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.274307:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274313:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe430951850394 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274328:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274333:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274339:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274345:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.274347:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274354:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274361:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f0cf83100) 00010000:00010000:0.0:1789302898.274364:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.274366:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274382:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185039b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274414:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185038d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a132c expref: 77 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274424:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.274426:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.274430:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185038d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a132c expref: 77 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.274440:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00000100:00100000:0.0:1789302898.274693:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33501:x1876219350569984:12345-192.168.203.11@tcp:101:cp.0 Request processed in 667us (855us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.274700:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1609 00000100:00100000:3.0:1789302898.278637:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.279126:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350570240 00000100:00100000:1.0:1789302898.279137:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350570240 00000100:00100000:1.0:1789302898.279196:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1610 00000100:00100000:1.0:1789302898.279206:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+76:33501:x1876219350570240:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302898.279226:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.279248:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe4309518503a2 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1333 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.279381:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.279397:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 113 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.279406:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.279439:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 113 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.279464:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 113 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.279478:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f756300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371185280:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302898.279600:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75b40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302898.291769:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f756300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371185280:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302898.291800:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.295898:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.295997:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350570368 00000100:00100000:0.0:1789302898.296007:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350570368 00010000:00010000:0.0:1789302898.296058:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 114 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.296098:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 466 00000100:00100000:0.0:1789302898.296107:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2348:x1876219350570368:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302898.296121:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302898.296128:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 114 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.296142:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 114 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.296162:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe43095185032b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1302 expref: 114 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.296179:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.296185:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.296200:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.296208:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.296241:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302898.296306:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:2348:x1876219350570368:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 200us (415us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.296314:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 466 00010000:00010000:2.0:1789302898.296391:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.296401:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.296423:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.296432:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.296440:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:2.0:1789302898.296969:0:5844:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x21:0x0], index 1: 0 00010000:00010000:2.0:1789302898.297136:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297146:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297155:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297186:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297194:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297200:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297205:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.297209:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297217:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297227:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.297230:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.297234:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297245:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518503a9 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:2.0:1789302898.297256:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dbcf7b2b reply state 00000000538b4af1 transno 4294968365 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.297264:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:2.0:1789302898.297297:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe4309518503a2 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 79 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297313:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 78 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.297322:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:2.0:1789302898.297324:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302898.297327:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc 0) 00000100:00100000:2.0:1789302898.297361:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cbb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33501:x1876219350570240:12345-192.168.203.11@tcp:101:cp.0 Request processed in 18158us (18734us total) trans 4294968365 rc 0/0 00000100:00100000:2.0:1789302898.297369:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1610 00010000:00010000:3.0:1789302898.299805:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.299817:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.299825:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.299830:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.299834:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.299842:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.299856:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f10d3c500) 00010000:00010000:3.0:1789302898.299859:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.299863:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.299875:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518503b0 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.302273:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.302336:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350570496 00000100:00100000:2.0:1789302898.302347:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350570496 00000100:00100000:2.0:1789302898.302421:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:2.0:1789302898.302432:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2345:x1876219350570496:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.302455:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.302478:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a133a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302898.302681:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x21:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302898.302701:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789302898.302706:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.302716:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.302734:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.302747:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc 0) 00000100:00100000:2.0:1789302898.302852:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2345:x1876219350570496:12345-192.168.203.11@tcp:101:cp.0 Request processed in 422us (594us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.302859:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 463 00000100:00100000:1.0:1789302898.306187:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.306280:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350570624 00000100:00100000:3.0:1789302898.306291:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350570624 00000100:00100000:3.0:1789302898.306385:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00100000:3.0:1789302898.306396:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:33501:x1876219350570624:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302898.306415:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.306434:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 2/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1341 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.306476:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.306485:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302898.306490:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302898.306495:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 4/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 20 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.306558:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.306572:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 20 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.306592:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371185408:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302898.306663:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a240 going to sleep for 11 seconds 00000100:00100000:3.0:1789302898.309786:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371185408:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302898.309820:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 3/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 20 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.309840:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b3000, rc 0) 00000100:00100000:3.0:1789302898.309930:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:33501:x1876219350570624:12345-192.168.203.11@tcp:101:cp.0 Request processed in 3535us (3749us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302898.309939:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 464 00000100:00100000:1.0:1789302898.309997:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.310126:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350570752 00000100:00100000:2.0:1789302898.310132:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350570752 00010000:00010000:2.0:1789302898.310161:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 20 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.310198:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 467 00000100:00100000:2.0:1789302898.310207:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350570752:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302898.310219:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302898.310239:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 20 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.310269:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 20 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.310291:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518503c5 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a133a expref: 20 pid: 6697 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.310305:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.310309:0:13636:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.310316:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 19 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.310327:0:13636:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40080000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 19 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.310357:0:13636:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 19 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.310369:0:13636:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371185536:192.168.203.11@tcp:105: 00000100:00100000:2.0:1789302898.311798:0:13636:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9ba80 going to sleep for 11 seconds 00010000:00010000:1.0:1789302898.319845:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302898.320220:0:13636:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_004:lustre-OST0001_UUID:13636:1876219371185536:192.168.203.11@tcp:105: 00010000:00010000:2.0:1789302898.320252:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302898.320464:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350570752:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 10256us (10465us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.320476:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 467 00000100:00100000:0.0:1789302898.322187:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.323289:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350571008 00000100:00100000:2.0:1789302898.323299:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350571008 00000100:00100000:2.0:1789302898.323396:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:2.0:1789302898.323408:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+78:33501:x1876219350571008:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302898.323862:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33501:x1876219350571008:12345-192.168.203.11@tcp:35:cp.0 Request processed in 455us (1679us total) trans 4294968366 rc 0/0 00000100:00100000:2.0:1789302898.323870:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 337 00000100:00100000:0.0:1789302898.360466:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.360753:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350571136 00000100:00100000:2.0:1789302898.360763:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350571136 00000100:00100000:0.0:1789302898.360931:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1611 00000100:00100000:0.0:1789302898.360942:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+113:33504:x1876219350571136:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302898.360963:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.360983:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518503d3 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1348 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361112:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361126:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361136:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361150:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361159:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361166:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361250:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518503e8 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361311:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518503e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361321:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518503e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302898.361353:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x34:0x0]: returning size 158 00000004:00200000:0.0:1789302898.361357:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302898.361384:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518503e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361396:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518503e8 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361405:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361412:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361419:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361428:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.361433:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361440:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361536:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.361539:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.361544:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361552:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518503da lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361567:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361574:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361579:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361585:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.361587:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361593:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361605:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f0cf83100) 00010000:00010000:0.0:1789302898.361607:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.361610:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361617:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518503e1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361649:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518503d3 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1348 expref: 115 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361664:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518503e8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1348 expref: 114 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.361671:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:0.0:1789302898.361673:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302898.361676:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b400, rc 0) 00000100:00100000:0.0:1789302898.361783:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33504:x1876219350571136:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 842us (1338us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.361790:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1611 00000100:00100000:3.0:1789302898.368149:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.368560:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350571264 00000100:00100000:1.0:1789302898.368570:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350571264 00000100:00100000:1.0:1789302898.368651:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 465 00000100:00100000:1.0:1789302898.368661:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350571264:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302898.368679:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.368710:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518503ef lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a134f expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.368767:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371185664:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302898.368855:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75d80 going to sleep for 11 seconds 00000100:00080000:2.0:1789302898.368920:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302898.371384:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.372454:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350571392 00000100:00100000:2.0:1789302898.372464:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350571392 00000100:00100000:2.0:1789302898.372546:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 83 00000100:00100000:2.0:1789302898.372557:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f07781850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2347:x1876219350571392:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:2.0:1789302898.372613:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.372634:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe4309518503f6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1356 expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372693:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372717:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372730:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372747:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372759:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372765:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372817:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372825:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372831:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372836:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.372841:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372849:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372861:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.372864:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.372870:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372876:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518503fd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372889:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372895:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372901:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372907:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.372909:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372914:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372921:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f10d3c500) 00010000:00010000:2.0:1789302898.372924:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.372927:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372933:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850404 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372960:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe4309518503f6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1356 expref: 115 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372970:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.372972:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789302898.372976:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe4309518503f6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1356 expref: 115 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.372997:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc -2) 00000100:00100000:2.0:1789302898.374867:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f07781850 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2347:x1876219350571392:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 2304us (3491us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302898.374880:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 83 00002000:00010000:1.0:1789302898.376030:0:8571:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb size: 0 -> 158 00002000:00010000:1.0:1789302898.376037:0:8571:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb mtime: 0 -> 1789302898 00002000:00010000:1.0:1789302898.376040:0:8571:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb atime: 0 -> 1789302898 00002000:00010000:1.0:1789302898.376042:0:8571:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x21:0x0] updating lvb ctime: 0 -> 1789302898 00000100:00100000:1.0:1789302898.376061:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8571:1876219371185664:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302898.376092:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518503ef lrc: 1/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a134f expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.376115:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe4309518503ef lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a134f expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.376129:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8c00, rc 0) 00000100:00100000:1.0:1789302898.376490:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350571264:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 7829us (8349us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302898.376498:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 465 00000100:00100000:1.0:1789302898.382272:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.382384:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350571520 00000100:00100000:0.0:1789302898.382393:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350571520 00000100:00100000:2.0:1789302898.382560:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1612 00000100:00100000:2.0:1789302898.382573:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33504:x1876219350571520:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302898.382591:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.382611:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185040b lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a138e expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.382677:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.382690:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.382779:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302898.382815:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:2.0:1789302898.382818:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:2.0:1789302898.382834:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.382859:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185040b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 116 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.382882:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 115 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.382889:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.382892:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1789302898.382896:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:2.0:1789302898.382999:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33504:x1876219350571520:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 429us (733us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.383007:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1612 00000100:00100000:0.0:1789302898.389117:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.389213:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350571648 00000100:00100000:2.0:1789302898.389222:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350571648 00000100:00100000:2.0:1789302898.389279:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 466 00000100:00100000:2.0:1789302898.389289:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350571648:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302898.389305:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.389323:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe430951850419 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1395 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.389393:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371185792:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302898.389543:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9ba80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302898.392346:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371185792:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302898.392395:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe430951850419 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1395 expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.392419:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe430951850419 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1395 expref: 18 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.392434:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc 0) 00000100:00100000:2.0:1789302898.392706:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350571648:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 3417us (3596us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.392716:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 466 00000100:00100000:1.0:1789302898.397755:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.397830:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350571904 00000100:00100000:2.0:1789302898.397840:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350571904 00000100:00100000:2.0:1789302898.397895:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1613 00000100:00100000:2.0:1789302898.397905:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33504:x1876219350571904:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:2.0:1789302898.398284:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].51, 136 off15128 00000100:00100000:2.0:1789302898.398386:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f756300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33504:x1876219350571904:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 481us (639us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.398395:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1613 00000100:00100000:1.0:1789302898.407103:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.407361:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350572032 00000100:00100000:0.0:1789302898.407381:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350572032 00000100:00100000:0.0:1789302898.407435:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1614 00000100:00100000:0.0:1789302898.407453:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33507:x1876219350572032:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302898.407477:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.407497:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850420 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a139c expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407605:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407619:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407629:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407645:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407653:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407660:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407705:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407713:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407719:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407726:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.407731:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407739:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407749:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.407752:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.407756:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407763:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850427 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407777:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407783:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407789:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407794:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.407797:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407804:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407811:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f0cf83100) 00010000:00010000:0.0:1789302898.407813:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.407816:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407822:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185042e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407852:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850420 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a139c expref: 116 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407862:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.407865:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.407869:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850420 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a139c expref: 116 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.407879:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00000100:00100000:0.0:1789302898.407927:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33507:x1876219350572032:12345-192.168.203.11@tcp:101:bash.0 Request processed in 483us (833us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.407934:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1614 00000100:00100000:1.0:1789302898.408895:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.409084:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350572160 00000100:00100000:0.0:1789302898.409091:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350572160 00000100:00100000:0.0:1789302898.409133:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 84 00000100:00100000:0.0:1789302898.409141:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:2347:x1876219350572160:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302898.409167:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.409181:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850435 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13a3 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409217:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409233:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409242:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409259:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409266:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409273:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409307:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409314:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409320:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409326:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.409330:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409338:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409348:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.409350:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.409354:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409361:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe43095185043c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409385:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409391:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409397:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409402:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.409405:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409412:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409420:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x71bff0c5 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.409423:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.409426:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409431:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850443 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409464:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850435 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13a3 expref: 116 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409474:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.409477:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302898.409481:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850435 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13a3 expref: 116 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.409491:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc -2) 00000100:00100000:0.0:1789302898.409564:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:2347:x1876219350572160:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 423us (670us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302898.409571:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 84 00000100:00100000:1.0:1789302898.418130:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.419241:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350572288 00000100:00100000:2.0:1789302898.419254:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350572288 00000100:00100000:2.0:1789302898.419699:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1615 00000100:00100000:2.0:1789302898.419710:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33508:x1876219350572288:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302898.419736:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.419757:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185044a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13aa expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419823:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419836:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419852:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419866:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419873:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419881:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419932:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419939:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419946:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419953:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.419958:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419966:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419978:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.419981:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.419986:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.419993:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850451 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420007:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420014:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420020:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420037:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.420041:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420047:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420054:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f10d3c500) 00010000:00010000:2.0:1789302898.420057:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.420059:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420066:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850458 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420099:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185044a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13aa expref: 80 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420115:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.420117:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.420120:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185044a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13aa expref: 80 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.420129:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc 0) 00000100:00100000:2.0:1789302898.420190:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33508:x1876219350572288:12345-192.168.203.11@tcp:101:rm.0 Request processed in 483us (2069us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.420197:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1615 00000100:00100000:3.0:1789302898.424228:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.424612:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350572416 00000100:00100000:0.0:1789302898.424624:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350572416 00000100:00100000:0.0:1789302898.424694:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1616 00000100:00100000:0.0:1789302898.424704:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33508:x1876219350572416:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302898.424724:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.424742:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095185045f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13b1 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424807:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424820:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424830:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424845:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424853:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424860:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424912:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424919:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424927:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424933:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.424937:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424945:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424957:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.424960:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.424964:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424971:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850466 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424983:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424991:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.424997:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.425002:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.425005:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.425011:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.425018:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.425021:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.425023:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.425029:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185046d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.425058:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095185045f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13b1 expref: 80 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.425068:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.425071:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.425074:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe43095185045f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13b1 expref: 80 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.425089:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:0.0:1789302898.425152:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33508:x1876219350572416:12345-192.168.203.11@tcp:101:rm.0 Request processed in 449us (931us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.425159:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1616 00000100:00100000:3.0:1789302898.438172:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.438425:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350572544 00000100:00100000:0.0:1789302898.438435:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350572544 00000100:00100000:0.0:1789302898.438493:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1617 00000100:00100000:0.0:1789302898.438541:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33509:x1876219350572544:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.438559:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.438581:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850474 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13b8 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438645:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438660:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438672:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438687:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438694:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438700:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438752:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438760:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438766:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438772:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.438777:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438785:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438795:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.438799:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.438803:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438810:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe43095185047b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438823:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438830:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438836:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438841:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.438845:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438851:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438857:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.438860:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.438862:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438868:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850482 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438900:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850474 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13b8 expref: 80 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438910:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.438912:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.438916:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850474 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13b8 expref: 80 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.438925:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:0.0:1789302898.438982:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa4700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33509:x1876219350572544:12345-192.168.203.11@tcp:101:cp.0 Request processed in 480us (817us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.438989:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1617 00000100:00100000:3.0:1789302898.441101:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.441279:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350572672 00000100:00100000:0.0:1789302898.441287:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350572672 00000100:00100000:0.0:1789302898.441533:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1618 00000100:00100000:0.0:1789302898.441544:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33509:x1876219350572672:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.441562:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.441580:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850489 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13bf expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441641:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441656:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441667:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441680:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441688:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441695:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441808:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441816:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441851:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441863:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.441867:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441875:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441887:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.441889:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.441893:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441897:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe430951850490 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441907:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441911:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441915:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441917:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.441919:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441922:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441929:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f0cf83100) 00010000:00010000:0.0:1789302898.441930:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.441932:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441935:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe430951850497 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441957:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850489 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13bf expref: 80 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441963:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.441964:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.441966:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850489 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13bf expref: 80 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.441972:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00000100:00100000:0.0:1789302898.442036:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33509:x1876219350572672:12345-192.168.203.11@tcp:101:cp.0 Request processed in 494us (940us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.442043:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1618 00000100:00100000:1.0:1789302898.449200:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.449285:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350572928 00000100:00100000:0.0:1789302898.449296:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350572928 00000100:00100000:0.0:1789302898.449388:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1619 00000100:00100000:0.0:1789302898.449398:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+79:33509:x1876219350572928:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.449418:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.449437:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185049e lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13c6 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.449589:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.449604:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 114 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.449613:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.449648:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 114 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.449663:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 114 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.449677:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371185920:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302898.449760:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302898.453713:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.453781:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350573056 00000100:00100000:0.0:1789302898.453788:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371185920:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302898.453789:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350573056 00010000:00010000:0.0:1789302898.453817:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.453832:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 115 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.453870:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 468 00000100:00100000:2.0:1789302898.453881:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:2347:x1876219350573056:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302898.453895:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302898.453901:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 115 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.453914:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 115 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.453933:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850412 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a138e expref: 115 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.453956:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.453961:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.453971:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.453980:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.454003:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302898.454019:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.454059:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2347:x1876219350573056:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 179us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.454067:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 468 00010000:00010000:0.0:1789302898.454090:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.454118:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.454130:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.454138:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302898.455887:0:18427:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x22:0x0], index 0: 0 00010000:00010000:1.0:1789302898.456127:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456142:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456152:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456189:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456196:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456203:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456209:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.456213:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456222:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456232:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.456234:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.456239:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456244:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518504a5 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302898.456346:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000005d37dabf reply state 00000000a3ff4e82 transno 4294968367 ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.456356:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302898.456811:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185049e lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 82 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456828:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 81 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.456835:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302898.456838:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302898.456840:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9000, rc 0) 00000100:00100000:1.0:1789302898.456965:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33509:x1876219350572928:12345-192.168.203.11@tcp:101:cp.0 Request processed in 7567us (7771us total) trans 4294968367 rc 0/0 00000100:00100000:1.0:1789302898.456974:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1619 00010000:00010000:0.0:1789302898.460403:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.460419:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.460427:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.460433:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.460438:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.460454:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.460470:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.460473:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.460477:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.460484:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504ac lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.462345:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.462410:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350573184 00000100:00100000:2.0:1789302898.462420:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350573184 00000100:00100000:2.0:1789302898.462482:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 467 00000100:00100000:2.0:1789302898.462493:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350573184:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.462525:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.462546:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13cd expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302898.462573:0:13439:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x22:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:2.0:1789302898.462593:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=735s v=(0 0) 00010000:00010000:2.0:1789302898.462597:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.462606:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.462628:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.462641:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc 0) 00000100:00100000:2.0:1789302898.462692:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2347:x1876219350573184:12345-192.168.203.11@tcp:101:cp.0 Request processed in 201us (355us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.462699:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 467 00000100:00100000:1.0:1789302898.466125:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.466193:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350573312 00000100:00100000:0.0:1789302898.466203:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350573312 00000100:00100000:0.0:1789302898.466252:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 468 00000100:00100000:0.0:1789302898.466262:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:33509:x1876219350573312:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.466279:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.466296:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 2/0,0 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13d4 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.466324:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.466334:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.466338:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.466343:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 4/0,0 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 18 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.466388:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.466402:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 18 pid: 13439 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.466417:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371186048:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302898.466519:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2540 going to sleep for 11 seconds 00000100:00100000:1.0:1789302898.472160:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.472229:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350573440 00000100:00100000:3.0:1789302898.472239:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350573440 00010000:00010000:3.0:1789302898.472285:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 19 pid: 13439 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.472328:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 469 00000100:00100000:3.0:1789302898.472337:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd04450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350573440:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302898.472350:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302898.472386:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 19 pid: 13439 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.472400:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 19 pid: 13439 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.472424:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302898.472428:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302898.472436:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 19 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.472447:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 19 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.472484:0:5830:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 19 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.472497:0:5830:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371186176:192.168.203.11@tcp:105: 00000100:00100000:3.0:1789302898.472599:0:5830:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a240 going to sleep for 11 seconds 00010000:00010000:0.0:1789302898.472761:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe4309518504c1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13cd expref: 19 pid: 13439 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.472783:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371186048:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302898.472805:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 18 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.472815:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3c00, rc 0) 00000100:00100000:0.0:1789302898.472840:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:33509:x1876219350573312:12345-192.168.203.11@tcp:101:cp.0 Request processed in 6580us (6723us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.472847:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 468 00000100:00100000:3.0:1789302898.477138:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.477406:0:5830:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33b80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-OST0000_UUID:5830:1876219371186176:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302898.477429:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.477486:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd04450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350573440:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 5149us (5333us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.477496:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 469 00000100:00100000:2.0:1789302898.478582:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350573696 00000100:00100000:2.0:1789302898.478591:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350573696 00000100:00100000:2.0:1789302898.478684:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:2.0:1789302898.478694:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+81:33509:x1876219350573696:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:2.0:1789302898.478999:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33509:x1876219350573696:12345-192.168.203.11@tcp:35:cp.0 Request processed in 305us (1868us total) trans 4294968368 rc 0/0 00000100:00100000:2.0:1789302898.479007:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 338 00000100:00100000:3.0:1789302898.511558:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.511790:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350573824 00000100:00100000:1.0:1789302898.511801:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350573824 00000100:00100000:1.0:1789302898.511860:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1620 00000100:00100000:1.0:1789302898.511870:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33512:x1876219350573824:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302898.511890:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.511909:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe4309518504cf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13db expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.511971:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.511985:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.511997:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512011:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512017:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512040:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512122:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518504e4 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512130:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518504e4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512137:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518504e4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302898.512168:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x35:0x0]: returning size 158 00000004:00200000:1.0:1789302898.512172:0:5844:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302898.512187:0:5844:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518504e4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512199:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518504e4 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512207:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512213:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512221:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512226:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.512231:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512239:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512250:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.512253:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.512268:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512275:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe4309518504d6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512288:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512294:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512301:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512306:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.512309:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512315:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512322:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f2c94d700) 00010000:00010000:1.0:1789302898.512324:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.512327:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512339:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe4309518504dd lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512365:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe4309518504cf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13db expref: 116 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512388:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518504e4 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13db expref: 115 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.512396:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302898.512398:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302898.512401:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8600, rc 0) 00000100:00100000:1.0:1789302898.512462:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33512:x1876219350573824:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 593us (914us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.512469:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1620 00000100:00100000:3.0:1789302898.515655:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.515699:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350573952 00000100:00100000:1.0:1789302898.515709:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350573952 00000100:00100000:1.0:1789302898.515764:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 469 00000100:00100000:1.0:1789302898.515774:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350573952:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302898.515846:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.515862:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518504eb lrc: 2/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13e2 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.516117:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371186304:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302898.516184:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75600 going to sleep for 11 seconds 00002000:00010000:2.0:1789302898.518089:0:18578:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x22:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302898.518096:0:18578:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x22:0x0] updating lvb mtime: 0 -> 1789302898 00002000:00010000:2.0:1789302898.518098:0:18578:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x22:0x0] updating lvb atime: 0 -> 1789302898 00002000:00010000:2.0:1789302898.518100:0:18578:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x22:0x0] updating lvb ctime: 0 -> 1789302898 00000100:00100000:2.0:1789302898.518119:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371186304:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302898.518148:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518504eb lrc: 1/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e2 expref: 17 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.518168:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518504eb lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e2 expref: 17 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.518183:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8000, rc 0) 00000100:00100000:2.0:1789302898.518227:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350573952:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2455us (2581us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.518236:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 469 00000100:00100000:1.0:1789302898.522224:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.522803:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350574080 00000100:00100000:0.0:1789302898.522814:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350574080 00000100:00100000:0.0:1789302898.522924:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1621 00000100:00100000:0.0:1789302898.522935:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33512:x1876219350574080:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302898.522955:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.522974:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504f2 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13e9 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.523058:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.523072:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.523084:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302898.523116:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:0.0:1789302898.523120:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302898.523135:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.523160:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518504f2 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 117 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.523177:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 116 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.523186:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.523189:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302898.523193:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2800, rc 0) 00000100:00100000:0.0:1789302898.523254:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33512:x1876219350574080:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 322us (1038us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.523261:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1621 00000100:00100000:3.0:1789302898.525808:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.525891:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350574208 00000100:00100000:1.0:1789302898.525900:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350574208 00000100:00100000:1.0:1789302898.525950:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 470 00000100:00100000:1.0:1789302898.525959:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2345:x1876219350574208:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302898.525974:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.525989:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850500 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13f0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.526184:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c93100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371186432:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302898.526256:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f759c0 going to sleep for 11 seconds 00000100:00100000:3.0:1789302898.528016:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c93100 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371186432:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302898.528056:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850500 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13f0 expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.528083:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850500 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13f0 expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.528096:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8200, rc 0) 00000100:00100000:3.0:1789302898.528182:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2345:x1876219350574208:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 2224us (2381us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.528190:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 470 00000100:00100000:3.0:1789302898.531756:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.532221:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350574464 00000100:00100000:1.0:1789302898.532231:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350574464 00000100:00100000:1.0:1789302898.532297:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1622 00000100:00100000:1.0:1789302898.532307:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33512:x1876219350574464:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:1.0:1789302898.532850:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].52, 136 off15264 00000100:00100000:1.0:1789302898.532931:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33512:x1876219350574464:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 624us (1182us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.532939:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1622 00000100:00100000:2.0:1789302898.540142:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.540296:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350574592 00000100:00100000:0.0:1789302898.540307:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350574592 00000100:00100000:0.0:1789302898.540366:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1623 00000100:00100000:0.0:1789302898.540385:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33513:x1876219350574592:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:0.0:1789302898.540404:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.540425:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850507 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13f7 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540498:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540555:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540572:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540632:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540641:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540647:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540699:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540707:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540714:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540720:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.540724:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540731:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540744:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.540747:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.540753:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540759:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185050e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540773:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540779:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540785:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540790:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.540793:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540799:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540807:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.540809:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.540812:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540818:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850515 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540851:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850507 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13f7 expref: 117 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540866:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.540869:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.540872:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850507 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13f7 expref: 117 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.540882:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:0.0:1789302898.540937:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33513:x1876219350574592:12345-192.168.203.11@tcp:101:bash.0 Request processed in 553us (801us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.540944:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1623 00000100:00100000:3.0:1789302898.555432:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.555550:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350574720 00000100:00100000:0.0:1789302898.555560:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350574720 00000100:00100000:0.0:1789302898.555614:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1624 00000100:00100000:0.0:1789302898.555624:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33514:x1876219350574720:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302898.555646:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.555665:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185051c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a13fe expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555723:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555736:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555748:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555761:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555769:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555775:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555824:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555832:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555839:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555844:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.555849:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555857:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555868:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.555871:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.555875:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555881:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850523 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555894:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555900:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555906:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555910:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.555913:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555919:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555932:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.555934:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.555937:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555943:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185052a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555973:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185051c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13fe expref: 83 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.555982:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.555984:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.555988:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe43095185051c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13fe expref: 83 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.556002:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc 0) 00000100:00100000:0.0:1789302898.556067:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33514:x1876219350574720:12345-192.168.203.11@tcp:101:rm.0 Request processed in 444us (643us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.556074:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1624 00000100:00100000:1.0:1789302898.562671:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.562933:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350574848 00000100:00100000:2.0:1789302898.562942:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350574848 00000100:00100000:2.0:1789302898.563027:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1625 00000100:00100000:2.0:1789302898.563037:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33514:x1876219350574848:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:2.0:1789302898.563059:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.563080:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850531 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1405 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563143:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563158:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563169:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563184:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563190:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563201:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563252:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563271:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563277:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563284:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.563288:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563296:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563315:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.563318:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.563322:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563334:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850538 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563348:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563355:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563362:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563367:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.563377:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563384:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563391:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f095b2800) 00010000:00010000:2.0:1789302898.563394:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.563396:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563403:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185053f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563433:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850531 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1405 expref: 83 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563443:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.563445:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.563448:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850531 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1405 expref: 83 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.563458:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517200, rc 0) 00000100:00100000:2.0:1789302898.563538:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33514:x1876219350574848:12345-192.168.203.11@tcp:101:rm.0 Request processed in 503us (876us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.563549:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1625 00000100:00100000:3.0:1789302898.581785:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.581910:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350574976 00000100:00100000:0.0:1789302898.581920:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350574976 00000100:00100000:0.0:1789302898.581980:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1626 00000100:00100000:0.0:1789302898.581990:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33515:x1876219350574976:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.582014:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.582099:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850546 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a140c expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582231:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582244:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582255:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582358:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582367:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582383:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582437:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582611:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582620:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582627:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.582632:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582640:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582652:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.582656:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.582661:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582667:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe43095185054d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582681:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582687:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582693:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582699:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.582701:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582706:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582713:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.582715:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.582717:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582723:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850554 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582757:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850546 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a140c expref: 83 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582767:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.582769:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.582773:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850546 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a140c expref: 83 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.582783:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc 0) 00000100:00100000:0.0:1789302898.582881:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33515:x1876219350574976:12345-192.168.203.11@tcp:101:cp.0 Request processed in 892us (1104us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.582889:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1626 00000100:00100000:1.0:1789302898.587182:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.587313:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350575104 00000100:00100000:2.0:1789302898.587323:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350575104 00000100:00100000:2.0:1789302898.587388:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1627 00000100:00100000:2.0:1789302898.587398:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33515:x1876219350575104:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.587415:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.587435:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185055b lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1413 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587548:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587563:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587574:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587587:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587594:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587601:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587653:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587665:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587672:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587678:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.587682:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587689:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587700:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.587703:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.587707:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587714:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850562 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587726:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587732:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587738:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587743:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.587746:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587752:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587758:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f095b2800) 00010000:00010000:2.0:1789302898.587760:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.587763:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587769:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850569 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587810:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185055b lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1413 expref: 83 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587820:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.587822:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.587826:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185055b lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1413 expref: 83 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.587836:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517200, rc 0) 00000100:00100000:2.0:1789302898.587993:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33515:x1876219350575104:12345-192.168.203.11@tcp:101:cp.0 Request processed in 596us (819us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.588001:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1627 00000100:00100000:1.0:1789302898.593419:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.593601:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350575360 00000100:00100000:2.0:1789302898.593610:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350575360 00000100:00100000:2.0:1789302898.593661:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1628 00000100:00100000:2.0:1789302898.593670:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+82:33515:x1876219350575360:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.593689:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.593708:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850570 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a141a expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.593817:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.593837:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 115 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.594025:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.594057:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 115 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.594070:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 115 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.594083:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371186560:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302898.594166:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302898.595822:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371186560:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302898.595853:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.599894:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.600004:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350575488 00000100:00100000:2.0:1789302898.600014:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350575488 00010000:00010000:2.0:1789302898.600064:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 116 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.600106:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 470 00000100:00100000:2.0:1789302898.600116:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:2347:x1876219350575488:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302898.600130:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302898.600138:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 116 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.600151:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 116 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.600171:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518504f9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13e9 expref: 116 pid: 5845 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.600189:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.600194:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.600203:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.600211:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.600245:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302898.600367:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.600386:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.600416:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.600426:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.600434:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:1.0:1789302898.600937:0:18427:0:(osp_object.c:1662:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x280000400:0x22:0x0], index 1: 0 00010000:00010000:1.0:1789302898.601086:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601096:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601104:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601130:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601137:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601143:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601149:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.601153:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601165:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601174:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.601177:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.601181:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601186:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850577 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302898.601196:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008c96112c reply state 00000000aa051412 transno 4294968369 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.601204:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:1.0:1789302898.601232:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850570 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 85 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601246:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 84 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.601253:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:1.0:1789302898.601255:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302898.601257:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8a00, rc 0) 00000100:00100000:2.0:1789302898.601348:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:2347:x1876219350575488:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1233us (1463us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302898.601357:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 470 00000100:00100000:1.0:1789302898.601538:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33515:x1876219350575360:12345-192.168.203.11@tcp:101:cp.0 Request processed in 7868us (8127us total) trans 4294968369 rc 0/0 00000100:00100000:1.0:1789302898.601546:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1628 00000100:00100000:1.0:1789302898.617891:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.617989:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.618014:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350575616 00000100:00100000:1.0:1789302898.618023:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350575616 00000100:00100000:1.0:1789302898.618064:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 471 00000100:00100000:1.0:1789302898.618073:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350575616:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302898.618090:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.618110:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1421 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302898.618135:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x22:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789302898.618155:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789302898.618159:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.618168:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 20 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.618185:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 20 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.618198:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8c00, rc 0) 00000100:00100000:1.0:1789302898.618244:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350575616:12345-192.168.203.11@tcp:101:cp.0 Request processed in 172us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.618250:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 471 00000100:00100000:3.0:1789302898.618355:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350575744 00000100:00100000:3.0:1789302898.618360:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350575744 00000100:00100000:3.0:1789302898.618402:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 85 00000100:00100000:3.0:1789302898.618409:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:2347:x1876219350575744:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:3.0:1789302898.618447:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.618460:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185059a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1428 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618522:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618535:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618545:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618562:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618569:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618575:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618624:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618631:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618638:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618644:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.618648:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618655:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618666:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.618670:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.618674:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618680:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe4309518505a1 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618692:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618699:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618760:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618766:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.618769:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.618775:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.619153:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.619163:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.619169:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.619174:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.619177:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.619184:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.619193:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f2c94d700) 00010000:00010000:0.0:1789302898.619195:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.619199:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.619205:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe43095185057e lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.619659:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f0cdf2400) 00010000:00010000:3.0:1789302898.619663:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.619667:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.619674:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe4309518505a8 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.619707:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185059a lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1428 expref: 85 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.619718:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.619720:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302898.619724:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe43095185059a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1428 expref: 85 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.619735:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc -2) 00000100:00100000:0.0:1789302898.620293:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+84:2347:x1876219350575744:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 1882us (2409us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302898.620303:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 85 00000100:00100000:1.0:1789302898.626961:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.627057:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350575872 00000100:00100000:2.0:1789302898.627066:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350575872 00000100:00100000:2.0:1789302898.627133:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 472 00000100:00100000:2.0:1789302898.627144:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:33515:x1876219350575872:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:2.0:1789302898.627164:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.627183:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 2/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1460 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.627216:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 21 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.627225:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.627229:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302898.627234:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 4/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 21 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.627265:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 21 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.627285:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 21 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.627310:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371186688:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302898.627410:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9b240 going to sleep for 11 seconds 00000100:00100000:2.0:1789302898.634475:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.634697:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350576000 00000100:00100000:1.0:1789302898.634707:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350576000 00010000:00010000:1.0:1789302898.634750:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 22 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.634807:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 471 00000100:00100000:1.0:1789302898.634817:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350576000:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302898.634833:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302898.634856:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 22 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.634871:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 22 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.634896:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.634900:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.634908:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 22 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.634919:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 22 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.634955:0:5831:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 22 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.634970:0:5831:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c93100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371186816:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302898.635060:0:5831:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75e40 going to sleep for 11 seconds 00010000:00010000:0.0:1789302898.635943:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007623b30d/0x64fe430951850593 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1421 expref: 22 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.635964:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe31c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371186688:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302898.635991:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 21 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.636003:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6a00, rc 0) 00000100:00100000:0.0:1789302898.636063:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+21:33515:x1876219350575872:12345-192.168.203.11@tcp:101:cp.0 Request processed in 8922us (9110us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.636071:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 472 00000100:00100000:1.0:1789302898.639622:0:5831:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c93100 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_002:lustre-OST0001_UUID:5831:1876219371186816:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302898.639648:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.639698:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2347:x1876219350576000:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4882us (5232us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.639709:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 471 00000100:00100000:3.0:1789302898.639955:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.640101:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350576256 00000100:00100000:0.0:1789302898.640110:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350576256 00000100:00100000:0.0:1789302898.640160:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:0.0:1789302898.640169:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+84:33515:x1876219350576256:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:0.0:1789302898.640533:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33515:x1876219350576256:12345-192.168.203.11@tcp:35:cp.0 Request processed in 364us (577us total) trans 4294968370 rc 0/0 00000100:00100000:0.0:1789302898.640541:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 339 00000100:00100000:3.0:1789302898.665231:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.665334:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350576384 00000100:00100000:2.0:1789302898.665344:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350576384 00000100:00100000:2.0:1789302898.665408:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 86 00000100:00100000:2.0:1789302898.665422:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:2348:x1876219350576384:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:2.0:1789302898.665484:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.665540:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe4309518505b6 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1467 expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665598:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665611:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665622:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665635:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665643:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665649:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665698:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665706:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665713:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665725:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.665729:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665737:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665748:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.665751:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.665756:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665762:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe4309518505bd lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665775:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665780:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665787:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665791:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302898.665793:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665799:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665810:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f095b2800) 00010000:00010000:2.0:1789302898.665813:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302898.665815:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665821:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe4309518505c4 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665847:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe4309518505b6 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1467 expref: 86 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665858:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302898.665860:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789302898.665863:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe4309518505b6 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1467 expref: 86 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.665872:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc -2) 00000100:00100000:2.0:1789302898.665928:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:2348:x1876219350576384:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 511us (707us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302898.665935:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 86 00000100:00100000:2.0:1789302898.682917:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.683249:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350576512 00000100:00100000:0.0:1789302898.683357:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350576512 00000100:00100000:0.0:1789302898.683578:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1629 00000100:00100000:0.0:1789302898.683589:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33522:x1876219350576512:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302898.683607:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.683625:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518505cb lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a149f expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683705:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683719:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683730:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683744:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683751:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683758:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683835:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518505e0 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683843:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518505e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683851:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518505e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302898.683881:0:6919:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x36:0x0]: returning size 158 00000004:00200000:0.0:1789302898.683885:0:6919:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302898.683900:0:6919:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518505e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683910:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518505e0 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683918:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683929:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683936:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683942:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.683946:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683954:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683965:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.683968:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.683973:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683980:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518505d2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683992:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.683998:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684004:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684010:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.684013:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684019:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684038:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.684040:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.684043:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684049:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518505d9 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684076:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe4309518505cb lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a149f expref: 117 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684089:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518505e0 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a149f expref: 116 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.684096:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:0.0:1789302898.684098:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302898.684101:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b000, rc 0) 00000100:00100000:0.0:1789302898.684159:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe31c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33522:x1876219350576512:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 573us (1251us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.684167:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1629 00000100:00100000:1.0:1789302898.686992:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.688146:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350576640 00000100:00100000:2.0:1789302898.688156:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350576640 00000100:00100000:2.0:1789302898.688243:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00100000:2.0:1789302898.688252:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350576640:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:2.0:1789302898.688277:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302898.688295:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b444b6a2/0x64fe4309518505e7 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14a6 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302898.688355:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371186944:192.168.203.11@tcp:106: 00000100:00100000:2.0:1789302898.688464:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3ff9ba80 going to sleep for 11 seconds 00002000:00010000:2.0:1789302898.696437:0:13439:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb size: 0 -> 158 00002000:00010000:2.0:1789302898.696457:0:13439:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb mtime: 0 -> 1789302898 00002000:00010000:2.0:1789302898.696460:0:13439:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb atime: 0 -> 1789302898 00002000:00010000:2.0:1789302898.696462:0:13439:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x280000400:0x22:0x0] updating lvb ctime: 0 -> 1789302898 00000100:00100000:2.0:1789302898.696486:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3a9e3800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371186944:192.168.203.11@tcp:106: 00010000:00010000:2.0:1789302898.696643:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b444b6a2/0x64fe4309518505e7 lrc: 1/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14a6 expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.696669:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b444b6a2/0x64fe4309518505e7 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14a6 expref: 18 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302898.696682:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc 0) 00000100:00100000:2.0:1789302898.696740:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350576640:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 8489us (9755us total) trans 0 rc 301/301 00000100:00100000:2.0:1789302898.696749:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00100000:0.0:1789302898.701188:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.701379:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371187200:0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701423:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302898.701458:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371187712:0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701480:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302898.701525:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302898.701531:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371187328:0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701550:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302898.701589:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371187712 00000100:00100000:1.0:1789302898.701595:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371187712 00000100:00100000:1.0:1789302898.701646:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 542 00000100:00100000:1.0:1789302898.701654:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371187712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701699:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371187712:12345-0@lo:400:kworker.0 Request processed in 46us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.701706:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 542 00000100:00100000:1.0:1789302898.701739:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371187712:0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701761:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350576768 00000100:00100000:1.0:1789302898.701764:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350576768 00000100:00100000:1.0:1789302898.701783:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371187200 00000100:00100000:1.0:1789302898.701785:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371187200 00000100:00100000:1.0:1789302898.701799:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371187328 00000100:00100000:1.0:1789302898.701801:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371187328 00000100:00100000:1.0:1789302898.701815:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:1.0:1789302898.701821:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371187200:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701850:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3300:x1876219371187200:12345-0@lo:400:kworker.0 Request processed in 29us (426us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.701856:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:1.0:1789302898.701874:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3300:1876219371187200:0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701890:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 518 00000100:00100000:1.0:1789302898.701896:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371187328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701923:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371187328:12345-0@lo:400:kworker.0 Request processed in 28us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.701928:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 518 00000100:00100000:1.0:1789302898.701946:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371187328:0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.701960:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1630 00000100:00100000:1.0:1789302898.701966:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33522:x1876219350576768:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302898.701977:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.701996:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe4309518505ee lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14ad expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.702076:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.702090:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.702101:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302898.702137:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:1.0:1789302898.702141:0:5843:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:1.0:1789302898.702157:0:5843:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.702180:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe4309518505ee lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 118 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.702197:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 117 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.702205:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.702208:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1789302898.702212:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8000, rc 0) 00000100:00100000:1.0:1789302898.702243:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33522:x1876219350576768:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 277us (1066us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.702249:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1630 00000100:00100000:0.0:1789302898.702324:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371187072:0@lo:400:kworker.0 00000100:00100000:0.0:1789302898.702354:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302898.702397:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02909500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371187584:0@lo:400:kworker.0 00000100:00100000:0.0:1789302898.702416:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302898.702431:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789302898.702435:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c47480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371187456:0@lo:400:kworker.0 00000100:00100000:0.0:1789302898.702452:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302898.702472:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371187584 00000100:00100000:0.0:1789302898.702476:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371187584 00000100:00100000:0.0:1789302898.702525:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 519 00000100:00100000:0.0:1789302898.702532:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371187584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302898.702564:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371187584:12345-0@lo:400:kworker.0 Request processed in 32us (146us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.702569:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 519 00000100:00100000:0.0:1789302898.702590:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02909500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371187584:0@lo:400:kworker.0 00000100:00080000:2.0:1789302898.702779:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302898.702784:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302898.704808:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371187072 00000100:00100000:1.0:1789302898.704820:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371187072 00000100:00100000:1.0:1789302898.704864:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:1.0:1789302898.704873:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371187072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.704951:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371187072:12345-0@lo:400:kworker.0 Request processed in 79us (2594us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.704957:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:1.0:1789302898.704988:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371187456 00000100:00100000:1.0:1789302898.704992:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371187456 00000100:00100000:1.0:1789302898.705024:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 543 00000100:00100000:1.0:1789302898.705030:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371187456:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.705062:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371187456:12345-0@lo:400:kworker.0 Request processed in 32us (2609us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.705069:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 543 00000100:00100000:1.0:1789302898.705114:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c47480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371187456:0@lo:400:kworker.0 00000100:00100000:1.0:1789302898.705123:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371187072:0@lo:400:kworker.0 00000100:00080000:2.0:1789302898.707316:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302898.709595:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.709738:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350576896 00000100:00100000:3.0:1789302898.709747:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350576896 00000100:00100000:3.0:1789302898.709794:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 474 00000100:00100000:3.0:1789302898.709805:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350576896:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302898.709822:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.709841:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe4309518505fc lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14b4 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.709897:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef79f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371187840:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302898.709967:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00a180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302898.714217:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef79f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371187840:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302898.714253:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe4309518505fc lrc: 1/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14b4 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.714277:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001b237177/0x64fe4309518505fc lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14b4 expref: 18 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.714291:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2600, rc 0) 00000100:00100000:1.0:1789302898.714347:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350576896:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 4544us (4760us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302898.714356:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 474 00000100:00100000:2.0:1789302898.721577:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.721980:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350577152 00000100:00100000:0.0:1789302898.722098:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350577152 00000100:00100000:0.0:1789302898.722165:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1631 00000100:00100000:0.0:1789302898.722178:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33522:x1876219350577152:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:0.0:1789302898.722576:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].53, 136 off15400 00000100:00100000:0.0:1789302898.722686:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33522:x1876219350577152:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 505us (1111us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.722696:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1631 00000001:00080000:2.0:1789302898.732050:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967384 is committed 00000100:00100000:0.0:1789302898.732416:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.732489:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350577280 00000100:00100000:3.0:1789302898.732530:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350577280 00000100:00100000:3.0:1789302898.732584:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1632 00000100:00100000:3.0:1789302898.732593:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33523:x1876219350577280:12345-192.168.203.11@tcp:101:bash.0 00010000:00010000:3.0:1789302898.732612:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.732630:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850603 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14bb expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732693:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732713:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732724:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732740:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732748:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732755:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732806:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732814:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732833:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732838:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b3c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.732843:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732851:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732863:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.732867:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.732872:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732878:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b3c00/0x64fe43095185060a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732891:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732897:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732903:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732908:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e028b2600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.732910:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732915:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732922:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f0cdf2400) 00010000:00010000:3.0:1789302898.732925:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.732927:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732933:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2600/0x64fe430951850611 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732962:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850603 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14bb expref: 118 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732973:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.732975:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.732978:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e028b2400/0x64fe430951850603 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14bb expref: 118 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.732987:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e028b2400, rc 0) 00000100:00100000:3.0:1789302898.733051:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33523:x1876219350577280:12345-192.168.203.11@tcp:101:bash.0 Request processed in 458us (639us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.733059:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1632 00000100:00100000:2.0:1789302898.740575:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302898.740611:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.740684:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350577408 00000100:00100000:3.0:1789302898.740694:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350577408 00000100:00100000:3.0:1789302898.740743:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1633 00000100:00100000:3.0:1789302898.740753:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33524:x1876219350577408:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302898.740772:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.740795:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850618 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14c2 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740877:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740890:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740903:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740917:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740924:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740929:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740980:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740987:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.740993:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741000:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f10e7fc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.741004:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741011:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741023:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.741025:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.741031:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741036:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7fc00/0x64fe43095185061f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741051:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741059:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741064:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741069:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f00030200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.741073:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741078:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741085:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302898.741087:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.741090:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741100:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030200/0x64fe430951850626 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741133:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850618 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14c2 expref: 86 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741144:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.741146:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.741149:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e000/0x64fe430951850618 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14c2 expref: 86 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.741163:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f10e7e000, rc 0) 00000100:00100000:3.0:1789302898.741228:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33524:x1876219350577408:12345-192.168.203.11@tcp:101:rm.0 Request processed in 477us (615us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.741235:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1633 00000100:00100000:2.0:1789302898.743704:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.743787:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350577664 00000100:00100000:3.0:1789302898.743797:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350577664 00000100:00100000:3.0:1789302898.743856:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1634 00000100:00100000:3.0:1789302898.743866:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33524:x1876219350577664:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:3.0:1789302898.743887:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.743906:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185062d lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14d0 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.743970:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.743984:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.743995:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744016:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744027:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744034:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744097:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744105:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744111:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744117:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.744125:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744136:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744147:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.744150:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.744155:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744161:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe430951850634 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744176:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744182:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744187:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744197:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.744204:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744210:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744216:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f0cdf2100) 00010000:00010000:3.0:1789302898.744219:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.744222:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744227:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe43095185063b lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744262:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185062d lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14d0 expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744272:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.744274:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.744278:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f00030000/0x64fe43095185062d lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14d0 expref: 86 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.744346:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f00030000, rc 0) 00000100:00100000:3.0:1789302898.747476:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33524:x1876219350577664:12345-192.168.203.11@tcp:101:rm.0 Request processed in 3598us (3766us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.747489:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1634 00000100:00100000:1.0:1789302898.747730:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350577536 00000100:00100000:1.0:1789302898.747740:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350577536 00000100:00100000:1.0:1789302898.747802:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 87 00000100:00100000:1.0:1789302898.747812:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:2348:x1876219350577536:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:1.0:1789302898.747848:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.747870:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850642 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14c9 expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.747927:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.747948:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.747959:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.747974:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.747982:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.747988:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748040:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748048:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748056:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748062:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.748067:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748075:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748087:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.748091:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.748097:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748104:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850649 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748117:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748123:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748129:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748134:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.748137:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748143:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748150:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xae1c5c33 (ffff960f2c94d100) 00010000:00010000:1.0:1789302898.748153:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.748156:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748162:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850650 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748193:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850642 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14c9 expref: 86 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748204:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.748206:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302898.748209:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850642 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14c9 expref: 86 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.748218:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc -2) 00000100:00100000:1.0:1789302898.751040:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:2348:x1876219350577536:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 3218us (10461us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302898.751054:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 87 00000100:00100000:2.0:1789302898.768437:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.768539:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350577792 00000100:00100000:1.0:1789302898.768550:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350577792 00000100:00100000:1.0:1789302898.768610:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1635 00000100:00100000:1.0:1789302898.768621:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33525:x1876219350577792:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302898.768646:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.768667:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850657 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14d7 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768730:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768745:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768756:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768770:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768778:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768785:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768838:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768846:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768853:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768859:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.768863:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768870:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768882:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.768885:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.768890:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768896:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe43095185065e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768911:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768917:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768923:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768928:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302898.768931:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768937:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768944:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f2c94d100) 00010000:00010000:1.0:1789302898.768946:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.768949:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768955:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9600/0x64fe430951850665 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768987:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850657 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14d7 expref: 86 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.768997:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.768999:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302898.769002:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850657 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14d7 expref: 86 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.769016:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc 0) 00000100:00100000:1.0:1789302898.769077:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33525:x1876219350577792:12345-192.168.203.11@tcp:101:cp.0 Request processed in 459us (649us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302898.769084:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1635 00000100:00100000:2.0:1789302898.772011:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.772093:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350577920 00000100:00100000:0.0:1789302898.772102:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350577920 00000100:00100000:0.0:1789302898.772153:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1636 00000100:00100000:0.0:1789302898.772161:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33525:x1876219350577920:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.772183:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.772200:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095185066c lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14de expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772268:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772282:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772293:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772309:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772316:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772331:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772395:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772404:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772415:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772420:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.772425:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772432:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772443:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.772446:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.772451:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772457:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850673 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772471:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772477:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772482:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772486:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.772489:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772495:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772544:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.772547:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.772550:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772555:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe43095185067a lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772586:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095185066c lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14de expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772596:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.772598:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.772601:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe43095185066c lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14de expref: 86 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.772610:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00000100:00100000:0.0:1789302898.772670:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33525:x1876219350577920:12345-192.168.203.11@tcp:101:cp.0 Request processed in 509us (667us total) trans 0 rc 301/301 00000100:00100000:0.0:1789302898.772677:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1636 00000100:00100000:0.0:1789302898.778943:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.779231:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350578176 00000100:00100000:3.0:1789302898.779242:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350578176 00000100:00100000:3.0:1789302898.779301:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1637 00000100:00100000:3.0:1789302898.779311:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+85:33525:x1876219350578176:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:3.0:1789302898.779331:0:6919:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.779352:0:6919:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850681 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14e5 expref: -99 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.779554:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.779569:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 116 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.779577:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 4/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.779611:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 116 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.779624:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 116 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.779637:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef79c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371187968:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302898.779723:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00acc0 going to sleep for 11 seconds 00000100:00100000:2.0:1789302898.784147:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.784175:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef79c00 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371187968:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302898.784201:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.784288:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350578304 00000100:00100000:1.0:1789302898.784295:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350578304 00010000:00010000:1.0:1789302898.784431:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 117 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.784572:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 472 00000100:00100000:1.0:1789302898.784583:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:2347:x1876219350578304:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302898.784600:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302898.784607:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 117 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.784619:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 117 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.784640:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe4309518505f5 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 7 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ad expref: 117 pid: 5843 timeout: 835 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.784654:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302898.784659:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.784668:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.784676:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 4/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.784738:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.784783:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:2347:x1876219350578304:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 202us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.784790:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 472 00010000:00010000:3.0:1789302898.786139:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.786151:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.786180:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.786191:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.786200:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00080000:3.0:1789302898.787000:0:6919:0:(osp_object.c:1662:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x240000400:0x23:0x0], index 0: 0 00010000:00010000:3.0:1789302898.787176:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe430951850696 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787187:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe430951850696 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787196:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe430951850696 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787229:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787236:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787242:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787249:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.787253:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787389:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787402:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.787405:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.787410:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787416:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe430951850688 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302898.787429:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 0000000093e8966b reply state 000000000b85950c transno 4294968371 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.787437:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00010000:00010000:3.0:1789302898.787782:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe430951850681 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14e5 expref: 88 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787799:0:6919:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0400/0x64fe430951850696 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14e5 expref: 87 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.787807:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302898.787809:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302898.787811:0:6919:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcf0400, rc 0) 00000100:00100000:3.0:1789302898.787960:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+88:33525:x1876219350578176:12345-192.168.203.11@tcp:101:cp.0 Request processed in 8650us (9029us total) trans 4294968371 rc 0/0 00000100:00100000:3.0:1789302898.787970:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1637 00010000:00010000:0.0:1789302898.790428:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.790454:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.790461:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.790467:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.790472:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.790480:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.790498:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f0cdf2100) 00010000:00010000:0.0:1789302898.790538:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.790544:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.790550:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe43095185068f lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.793864:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.793933:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350578432 00000100:00100000:1.0:1789302898.793944:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350578432 00000100:00100000:1.0:1789302898.794006:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 475 00000100:00100000:1.0:1789302898.794017:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350578432:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:1.0:1789302898.794044:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.794065:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14ec expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302898.794093:0:8571:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x23:0x0] initial LVB size: 0, mtime: 0x0, atime: 0x0, ctime: 0x0, blocks: 0x1 00010000:00010000:1.0:1789302898.794114:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=735s v=(0 0) 00010000:00010000:1.0:1789302898.794118:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.794127:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.794143:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 18 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.794155:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8000, rc 0) 00000100:00100000:1.0:1789302898.794218:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350578432:12345-192.168.203.11@tcp:101:cp.0 Request processed in 203us (363us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.794226:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 475 00000100:00100000:2.0:1789302898.798992:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.799176:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350578560 00000100:00100000:0.0:1789302898.799188:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350578560 00000100:00100000:0.0:1789302898.799246:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 476 00000100:00100000:0.0:1789302898.799256:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:33525:x1876219350578560:12345-192.168.203.11@tcp:101:cp.0 00010000:00010000:0.0:1789302898.799273:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.799291:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 2/0,0 mode: --/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14f3 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.799322:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 19 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.799331:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.799335:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302898.799341:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: --/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->4095] (req 0->4095) gid 0 flags: 0x50000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 19 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.799390:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 19 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.799404:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 19 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.799418:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371188096:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302898.799533:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2d80 going to sleep for 11 seconds 00000100:00100000:2.0:1789302898.807094:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.807177:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350578688 00000100:00100000:1.0:1789302898.807187:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350578688 00010000:00010000:1.0:1789302898.807224:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 20 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.807260:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00100000:1.0:1789302898.807268:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2347:x1876219350578688:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302898.807281:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302898.807306:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 20 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.807320:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 20 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.807344:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.807348:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302898.807357:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 20 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.807376:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50080000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 20 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.807416:0:5829:0:(ldlm_lockd.c:1082:ldlm_server_completion_ast()) ### server preparing completion AST ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 20 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.807436:0:5829:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371188224:192.168.203.11@tcp:105: 00000100:00100000:1.0:1789302898.807538:0:5829:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75780 going to sleep for 11 seconds 00010000:00010000:0.0:1789302898.807812:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006842f657/0x64fe4309518506a4 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14ec expref: 20 pid: 8571 timeout: 835 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302898.807895:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70ce00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371188096:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302898.807918:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 19 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.807929:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345aa00, rc 0) 00000100:00100000:0.0:1789302898.807972:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:33525:x1876219350578560:12345-192.168.203.11@tcp:101:cp.0 Request processed in 8718us (8992us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.807980:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 476 00000100:00100000:1.0:1789302898.811290:0:5829:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_000:lustre-OST0000_UUID:5829:1876219371188224:192.168.203.11@tcp:105: 00010000:00010000:1.0:1789302898.811314:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302898.811366:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2347:x1876219350578688:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 4097us (4280us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302898.811387:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 473 00000100:00100000:2.0:1789302898.812274:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.812350:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350578944 00000100:00100000:3.0:1789302898.812358:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350578944 00000100:00100000:3.0:1789302898.812417:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:3.0:1789302898.812426:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+87:33525:x1876219350578944:12345-192.168.203.11@tcp:35:cp.0 00000100:00100000:3.0:1789302898.813174:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+88:33525:x1876219350578944:12345-192.168.203.11@tcp:35:cp.0 Request processed in 746us (900us total) trans 4294968372 rc 0/0 00000100:00100000:3.0:1789302898.813183:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 340 00000100:00100000:2.0:1789302898.857299:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.857436:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350579072 00000100:00100000:3.0:1789302898.857453:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350579072 00000100:00100000:3.0:1789302898.857551:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1638 00000100:00100000:3.0:1789302898.857562:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33528:x1876219350579072:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302898.857582:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.857603:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe4309518506b2 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a14fa expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857680:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857696:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857708:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857724:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857735:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857742:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857835:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518506c7 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857845:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518506c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857852:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518506c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302898.857886:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x37:0x0]: returning size 158 00000004:00200000:3.0:1789302898.857890:0:18427:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 1 00000004:00010000:3.0:1789302898.857907:0:18427:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518506c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857919:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518506c7 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857928:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857934:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857940:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857949:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf1800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.857953:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857960:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857972:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302898.857975:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.857981:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.857987:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1800/0x64fe4309518506b9 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858001:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858007:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858012:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858017:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcf0600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302898.858020:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858026:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858037:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6f5308b8 (ffff960f0cdf2400) 00010000:00010000:3.0:1789302898.858040:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302898.858042:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858047:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0600/0x64fe4309518506c0 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x6f5308b8 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858078:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1a00/0x64fe4309518506b2 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14fa expref: 118 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858092:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120ddc00/0x64fe4309518506c7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14fa expref: 117 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.858099:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302898.858101:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302898.858104:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f120ddc00, rc 0) 00000100:00100000:3.0:1789302898.858167:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcddf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33528:x1876219350579072:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 607us (879us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302898.858174:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1638 00000100:00100000:2.0:1789302898.865273:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.865433:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350579328 00000100:00100000:0.0:1789302898.865450:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350579328 00000100:00100000:0.0:1789302898.865546:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 88 00000100:00100000:0.0:1789302898.865558:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:2345:x1876219350579328:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302898.865614:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.865643:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518506ce lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1508 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865699:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865714:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865725:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865740:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865747:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865753:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865804:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865811:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865817:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865823:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.865828:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865836:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865848:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.865851:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.865855:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865861:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518506d5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865874:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865880:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865885:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865889:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302898.865892:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865897:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865904:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xbb834312 (ffff960f0cf83e00) 00010000:00010000:0.0:1789302898.865906:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302898.865921:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865926:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865954:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518506ce lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1508 expref: 118 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865963:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.865966:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302898.865969:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe4309518506ce lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1508 expref: 118 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.865978:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc -2) 00000100:00100000:0.0:1789302898.866035:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:2345:x1876219350579328:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 479us (785us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302898.866043:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 88 00000100:00100000:0.0:1789302898.868226:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.868535:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350579200 00000100:00100000:3.0:1789302898.868546:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350579200 00000100:00100000:3.0:1789302898.868600:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 477 00000100:00100000:3.0:1789302898.868610:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350579200:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:3.0:1789302898.868626:0:13439:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302898.868750:0:13439:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518506e3 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1501 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302898.868815:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef79c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371188352:192.168.203.11@tcp:106: 00000100:00100000:3.0:1789302898.868962:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00acc0 going to sleep for 11 seconds 00002000:00010000:3.0:1789302898.874826:0:13439:0:(ofd_lvb.c:260:ofd_lvbo_update()) res: [0x240000400:0x23:0x0] updating lvb size: 0 -> 158 00002000:00010000:3.0:1789302898.874836:0:13439:0:(ofd_lvb.c:266:ofd_lvbo_update()) res: [0x240000400:0x23:0x0] updating lvb mtime: 0 -> 1789302898 00002000:00010000:3.0:1789302898.874839:0:13439:0:(ofd_lvb.c:272:ofd_lvbo_update()) res: [0x240000400:0x23:0x0] updating lvb atime: 0 -> 1789302898 00002000:00010000:3.0:1789302898.874841:0:13439:0:(ofd_lvb.c:278:ofd_lvbo_update()) res: [0x240000400:0x23:0x0] updating lvb ctime: 0 -> 1789302898 00000100:00100000:3.0:1789302898.874866:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef79c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371188352:192.168.203.11@tcp:106: 00010000:00010000:3.0:1789302898.874904:0:13439:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518506e3 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1501 expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.874937:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518506e3 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1501 expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302898.874952:0:13439:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00000100:00100000:3.0:1789302898.875012:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2348:x1876219350579200:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 6403us (6791us total) trans 0 rc 301/301 00000100:00100000:3.0:1789302898.875028:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 477 00000100:00100000:2.0:1789302898.880572:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302898.880666:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350579456 00000100:00100000:0.0:1789302898.880675:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350579456 00000100:00100000:0.0:1789302898.880735:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1639 00000100:00100000:0.0:1789302898.880745:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33528:x1876219350579456:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:0.0:1789302898.880764:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302898.880784:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506ea lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1540 expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.880855:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518506f1 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.880874:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518506f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.880885:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518506f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302898.880932:0:5845:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000401:0x53:0x0]: returning size 16384 00000004:00200000:0.0:1789302898.880936:0:5845:0:(mdt_handler.c:1574:mdt_getattr_internal()) dirent count 18446744073709551615 stripe count 1 MDT count 1 00000004:00010000:0.0:1789302898.880953:0:5845:0:(mdt_handler.c:2507:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518506f1 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.880979:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe4309518506ea lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1540 expref: 119 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.880998:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518506f1 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1540 expref: 118 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302898.881008:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302898.881011:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1789302898.881015:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf2a00, rc 0) 00000100:00100000:0.0:1789302898.881083:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcded80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:33528:x1876219350579456:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 339us (520us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302898.881090:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1639 00000100:00100000:0.0:1789302898.885253:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302898.885397:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350579584 00000100:00100000:1.0:1789302898.885406:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350579584 00000100:00100000:1.0:1789302898.885473:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 478 00000100:00100000:1.0:1789302898.885483:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350579584:12345-192.168.203.11@tcp:101:lfs.0 00010000:00010000:1.0:1789302898.885526:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302898.885545:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe4309518506f8 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1547 expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302898.885602:0:8571:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371188480:192.168.203.11@tcp:106: 00000100:00100000:1.0:1789302898.886901:0:8571:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302898.890218:0:8571:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290ad80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0000_UUID:8571:1876219371188480:192.168.203.11@tcp:106: 00010000:00010000:1.0:1789302898.890323:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe4309518506f8 lrc: 1/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1547 expref: 17 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.890348:0:8571:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe4309518506f8 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1547 expref: 17 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302898.890380:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc 0) 00000100:00100000:1.0:1789302898.890597:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2347:x1876219350579584:12345-192.168.203.11@tcp:101:lfs.0 Request processed in 5113us (5351us total) trans 0 rc 301/301 00000100:00100000:1.0:1789302898.890606:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 478 00000100:00100000:0.0:1789302898.901583:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302898.901782:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350579840 00000100:00100000:3.0:1789302898.901792:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350579840 00000100:00100000:3.0:1789302898.901885:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1640 00000100:00100000:3.0:1789302898.901896:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:33528:x1876219350579840:12345-192.168.203.11@tcp:58:lfs.0 00000040:00080000:3.0:1789302898.902425:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x8:0x0].54, 136 off15536 00000100:00100000:3.0:1789302898.902530:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:33528:x1876219350579840:12345-192.168.203.11@tcp:58:lfs.0 Request processed in 635us (955us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302898.902541:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1640 00000100:00100000:1.0:1789302899.213435:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371188608:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302899.213548:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302899.213620:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371188608 00000100:00100000:0.0:1789302899.213629:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371188608 00000100:00100000:0.0:1789302899.213691:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:0.0:1789302899.213702:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371188608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302899.213725:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 95223808 pending 0 free 3856662528 avail 3851419648 00000100:00100000:0.0:1789302899.213782:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290ad80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371188608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (233us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.213789:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1789302899.213880:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70dc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371188608:0@lo:13:osp-pre-1-0.0 00000040:00080000:1.0:1789302899.277580:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302899.277588:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302899.277634:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 53 count 13 00000040:00080000:1.0:1789302899.277642:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 1 count 2 00000100:00100000:1.0:1789302899.278008:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b800 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371188736:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302899.278098:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f759c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302899.304435:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.304478:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.304527:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.304553:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350579968 00000100:00100000:1.0:1789302899.304562:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350579968 00000100:00100000:1.0:1789302899.304613:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350580096 00000100:00100000:1.0:1789302899.304617:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350580096 00000100:00100000:1.0:1789302899.304635:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350580224 00000100:00100000:1.0:1789302899.304637:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350580224 00000100:00100000:1.0:1789302899.304650:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1641 00000100:00100000:1.0:1789302899.304659:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33540:x1876219350579968:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302899.304697:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x33:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302899.304804:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33540:x1876219350579968:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 145us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.304812:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1641 00000100:00100000:1.0:1789302899.304830:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1642 00000100:00100000:1.0:1789302899.304836:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+90:33541:x1876219350580096:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302899.304847:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x34:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302899.304883:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+90:33541:x1876219350580096:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 46us (404us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.304889:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1642 00000100:00100000:1.0:1789302899.304901:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1643 00000100:00100000:1.0:1789302899.304908:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+89:33542:x1876219350580224:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302899.304919:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302899.304953:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02908380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+89:33542:x1876219350580224:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 46us (451us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.304959:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1643 00000100:00100000:0.0:1789302899.305308:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b800 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371188736:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:0.0:1789302899.319257:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.319300:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.319383:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350580480 00000100:00100000:1.0:1789302899.319393:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350580480 00000100:00100000:1.0:1789302899.319544:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350580736 00000100:00100000:1.0:1789302899.319548:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350580736 00000100:00100000:1.0:1789302899.319570:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1644 00000100:00100000:1.0:1789302899.319579:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+89:33540:x1876219350580480:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.319600:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302899.319621:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518506ff lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a154e expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.319708:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850706 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.319724:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850706 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.319738:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850706 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:1.0:1789302899.319749:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x33:0x0]: returning size 158 00010000:00010000:1.0:1789302899.319887:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe4309518506ff lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a154e expref: 92 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.319903:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850706 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a154e expref: 91 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.319911:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:1.0:1789302899.319914:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1789302899.319918:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc 0) 00000100:00100000:2.0:1789302899.320384:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1645 00000100:00100000:2.0:1789302899.320395:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33542:x1876219350580736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302899.320405:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302899.320415:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850714 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1555 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.321248:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33540:x1876219350580480:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 1664us (1995us total) trans 4294968373 rc 0/0 00000100:00100000:1.0:1789302899.321256:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1644 00010000:00010000:2.0:1789302899.322372:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185071b lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.322388:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185071b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.322399:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185071b lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:2.0:1789302899.322409:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x35:0x0]: returning size 158 00010000:00010000:2.0:1789302899.324896:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6600/0x64fe430951850714 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1555 expref: 93 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.324925:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185071b lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1555 expref: 92 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.324935:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:2.0:1789302899.324949:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1789302899.324953:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7c00, rc 0) 00000100:00100000:2.0:1789302899.332319:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350580736:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 11915us (13006us total) trans 4294968374 rc 0/0 00000100:00100000:2.0:1789302899.332337:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1645 00000100:00100000:1.0:1789302899.338304:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.338809:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.338844:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.338897:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350581120 00000100:00100000:1.0:1789302899.338906:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350581120 00000100:00100000:1.0:1789302899.338956:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 479 00000100:00100000:1.0:1789302899.338966:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350581120:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.339012:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.339039:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 18 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.339048:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x21:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302899.339052:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.339058:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.339092:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 18 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.339107:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 18 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.339121:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371188864:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302899.339198:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0c76f900 going to sleep for 11 seconds 00000100:00100000:1.0:1789302899.339216:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350581248 00000100:00100000:1.0:1789302899.339220:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350581248 00000100:00100000:1.0:1789302899.339254:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 76 00000100:00100000:1.0:1789302899.339262:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:2348:x1876219350581248:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.339273:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.339296:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850706 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a154e expref: 92 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.339310:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:1.0:1789302899.339315:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.339342:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:2348:x1876219350581248:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 80us (497us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.339350:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 76 00000100:00100000:3.0:1789302899.342641:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350580992 00000100:00100000:3.0:1789302899.342652:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350580992 00000100:00100000:3.0:1789302899.342720:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1646 00000100:00100000:3.0:1789302899.342730:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33541:x1876219350580992:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302899.342749:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302899.342768:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850730 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a155c expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.342872:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850737 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.342886:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850737 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.342898:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850737 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302899.342909:0:5844:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x34:0x0]: returning size 158 00010000:00010000:3.0:1789302899.343103:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850730 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a155c expref: 94 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.343123:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850737 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a155c expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.343132:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:3.0:1789302899.343136:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302899.343139:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc 0) 00000100:00100000:3.0:1789302899.343212:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+93:33541:x1876219350580992:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 484us (4922us total) trans 4294968375 rc 0/0 00000100:00100000:3.0:1789302899.343219:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1646 00010000:00010000:2.0:1789302899.344706:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302899.345109:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.345135:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.345183:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350581504 00000100:00100000:2.0:1789302899.345191:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350581504 00000100:00100000:2.0:1789302899.345244:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 77 00000100:00100000:2.0:1789302899.345255:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+93:2347:x1876219350581504:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302899.345270:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302899.345293:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe43095185071b lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1555 expref: 93 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.345308:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:2.0:1789302899.345313:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302899.345356:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:2347:x1876219350581504:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 104us (219us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.345362:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 77 00000100:00100000:1.0:1789302899.346165:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350581376 00000100:00100000:1.0:1789302899.346173:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350581376 00000100:00100000:1.0:1789302899.346209:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 480 00000100:00100000:1.0:1789302899.346217:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350581376:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.346252:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.346269:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 19 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.346278:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x22:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302899.346282:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.346287:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.346320:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 19 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.346332:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 19 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.346347:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371188992:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302899.346424:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f09567f00 going to sleep for 11 seconds 00000100:00100000:1.0:1789302899.346474:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290a680 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371188864:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302899.346494:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.346571:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.346651:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350581760 00000100:00100000:1.0:1789302899.346655:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350581760 00000100:00100000:1.0:1789302899.346684:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 78 00000100:00100000:1.0:1789302899.346692:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd07050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:2345:x1876219350581760:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.346700:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.346717:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850737 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a155c expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.346731:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:1.0:1789302899.346735:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.346758:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd07050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:2345:x1876219350581760:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 68us (187us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.346764:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 78 00000100:00100000:1.0:1789302899.350824:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.350924:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350581632 00000100:00100000:2.0:1789302899.350936:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350581632 00002000:00100000:2.0:1789302899.350984:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x21:0x0] (0->157) req@ffff960e0290a680 x1876219350581632/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302899.350996:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0290a680 with x1876219350581632 ext(0->157) 00010000:00010000:2.0:1789302899.351004:0:18429:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 20 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.351096:0:18429:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 20 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302899.351105:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960e0290a680 00000100:00100000:2.0:1789302899.351174:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 18 00000100:00100000:2.0:1789302899.351184:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350581632:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:2.0:1789302899.352001:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2348:x1876219350581632:12345-192.168.203.11@tcp:4:cp.0 Request processed in 815us (1184us total) trans 4294967387 rc 0/0 00000100:00100000:2.0:1789302899.352010:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 18 00002000:00100000:2.0:1789302899.352027:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x21:0x0] (0->157) req@ffff960e0290a680 x1876219350581632/t4294967387(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/448 e 0 to 0 dl 1789302910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:2.0:1789302899.352036:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e0290a680 with x1876219350581632 ext(0->157) 00010000:00010000:2.0:1789302899.352044:0:18429:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 22 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.352054:0:18429:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 22 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302899.352061:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960e0290a680 00000100:00100000:2.0:1789302899.357267:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.357349:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350581888 00000100:00100000:0.0:1789302899.357359:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350581888 00000100:00100000:0.0:1789302899.357425:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 481 00000100:00100000:0.0:1789302899.357435:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+14:2347:x1876219350581888:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.357478:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.357498:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 14 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.357541:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x21:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789302899.357546:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302899.357551:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 4/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.357588:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 14 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.357600:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 14 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.357614:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371189120:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302899.357692:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2180 going to sleep for 11 seconds 00000100:00100000:3.0:1789302899.358453:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.358481:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350582016 00000100:00100000:3.0:1789302899.358486:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350582016 00002000:00100000:3.0:1789302899.358543:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x22:0x0] (0->157) req@ffff960f3ef79c00 x1876219350582016/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.358559:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef79c00 with x1876219350582016 ext(0->157) 00010000:00010000:3.0:1789302899.358571:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 22 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.358583:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 22 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302899.358589:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3ef79c00 00000100:00100000:3.0:1789302899.358609:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 19 00000100:00100000:3.0:1789302899.358617:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2346:x1876219350582016:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:3.0:1789302899.359082:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2346:x1876219350582016:12345-192.168.203.11@tcp:4:cp.0 Request processed in 464us (629us total) trans 4294967388 rc 0/0 00000100:00100000:3.0:1789302899.359090:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 19 00002000:00100000:3.0:1789302899.359097:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x22:0x0] (0->157) req@ffff960f3ef79c00 x1876219350582016/t4294967388(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/448 e 0 to 0 dl 1789302910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.359110:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ef79c00 with x1876219350582016 ext(0->157) 00010000:00010000:3.0:1789302899.359115:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 24 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.359123:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 5/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 24 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302899.359129:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3ef79c00 00000100:00100000:0.0:1789302899.359647:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290aa00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:6695:1876219371188992:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302899.359671:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302899.373804:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.376592:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350582144 00000100:00100000:1.0:1789302899.376608:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350582144 00010000:00010000:1.0:1789302899.376659:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 24 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.376705:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 474 00000100:00100000:1.0:1789302899.376715:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2345:x1876219350582144:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.376728:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.376750:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 24 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.376763:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 24 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.376784:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe4309518502b4 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d1 expref: 24 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.376801:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.376806:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.376812:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.376821:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.376859:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.376922:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:2345:x1876219350582144:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 208us (3125us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.376930:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 474 00010000:00010000:3.0:1789302899.377277:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377287:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377314:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377323:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377331:0:18578:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377337:0:18578:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302899.377342:0:18578:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377352:0:18578:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377374:0:18578:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.377382:0:18578:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000af4abfe8/0x64fe430951850729 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.377537:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:2348:x1876219350581120:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 38571us (38728us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.377545:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 479 00000100:00100000:1.0:1789302899.381172:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.381209:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.382214:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350582400 00000100:00100000:1.0:1789302899.382226:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350582400 00002000:00100000:1.0:1789302899.382280:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x21:0x0] (0->157) req@ffff960e02909880 x1876219350582400/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302899.382291:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02909880 with x1876219350582400 ext(0->157) 00010000:00010000:1.0:1789302899.382300:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 15 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.382314:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 15 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302899.382322:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960e02909880 00000100:00100000:1.0:1789302899.382361:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 20 00000100:00100000:1.0:1789302899.382381:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2347:x1876219350582400:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302899.382914:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2347:x1876219350582400:12345-192.168.203.11@tcp:4:cp.0 Request processed in 534us (1751us total) trans 4294967385 rc 0/0 00000100:00100000:1.0:1789302899.382923:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 20 00002000:00100000:1.0:1789302899.382930:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x21:0x0] (0->157) req@ffff960e02909880 x1876219350582400/t4294967385(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/448 e 0 to 0 dl 1789302910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302899.382938:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960e02909880 with x1876219350582400 ext(0->157) 00010000:00010000:1.0:1789302899.382945:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 17 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.382955:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 17 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302899.382968:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960e02909880 00000100:00100000:3.0:1789302899.383411:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70c700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371189120:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302899.383435:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.383695:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350582272 00000100:00100000:3.0:1789302899.383701:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350582272 00010000:00010000:3.0:1789302899.383728:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 22 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.383764:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 475 00000100:00100000:3.0:1789302899.383772:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350582272:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302899.383783:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302899.383804:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 22 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.383816:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 22 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.383838:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518504c8 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13d4 expref: 22 pid: 6696 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.383851:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302899.383856:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:3.0:1789302899.383861:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.383871:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.383894:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302899.383921:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+21:2347:x1876219350582272:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 151us (2711us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.383929:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 475 00010000:00010000:0.0:1789302899.384335:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384347:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384382:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384391:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384400:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384413:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8e00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302899.384417:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384426:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384438:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.384446:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e41f0adc/0x64fe430951850745 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.384878:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+20:2348:x1876219350581376:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 38425us (39536us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.384897:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 480 00000100:00100000:3.0:1789302899.400317:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.400388:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.400683:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350582528 00000100:00100000:1.0:1789302899.400693:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350582528 00010000:00010000:1.0:1789302899.400735:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 17 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.400772:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 476 00000100:00100000:1.0:1789302899.400781:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2347:x1876219350582528:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.400797:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.400822:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 17 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.400836:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 17 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.400859:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c810bd51/0x64fe4309518503cc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1341 expref: 17 pid: 6695 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.400875:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.400879:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.400884:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.400895:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.400927:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.400972:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350582528:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 192us (664us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.400979:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 476 00010000:00010000:1.0:1789302899.401178:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401189:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401216:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401224:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401232:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401238:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302899.401241:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401250:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401268:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.401276:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185074c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.401321:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2347:x1876219350581888:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 43888us (44061us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.401329:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 481 00000100:00100000:1.0:1789302899.402496:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.402540:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350582784 00000100:00100000:1.0:1789302899.402548:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350582784 00000100:00100000:1.0:1789302899.402586:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350583040 00000100:00100000:1.0:1789302899.402589:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350583040 00000100:00100000:1.0:1789302899.402611:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1647 00000100:00100000:1.0:1789302899.402619:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350582784:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302899.402707:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350582784:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 89us (2318us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.402715:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1647 00000100:00100000:1.0:1789302899.402730:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1648 00000100:00100000:1.0:1789302899.402736:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33540:x1876219350583040:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:1.0:1789302899.402770:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33540:x1876219350583040:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 34us (280us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.402776:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1648 00000100:00100000:0.0:1789302899.409977:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.424067:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350583296 00000100:00100000:3.0:1789302899.424082:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350583296 00000100:00100000:3.0:1789302899.424137:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1649 00000100:00100000:3.0:1789302899.424146:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33541:x1876219350583296:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302899.424666:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33541:x1876219350583296:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 519us (14695us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.424675:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1649 00000100:00100000:0.0:1789302899.429976:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.430110:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.430717:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350583424 00000100:00100000:2.0:1789302899.430728:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350583424 00000100:00100000:2.0:1789302899.430774:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350583552 00000100:00100000:2.0:1789302899.430777:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350583552 00000100:00100000:2.0:1789302899.430830:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 482 00000100:00100000:2.0:1789302899.430841:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+21:2347:x1876219350583424:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302899.430859:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302899.430880:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1563 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302899.430908:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x22:0x0] initial LVB size: 158, mtime: 0x6aa69872, atime: 0x0, ctime: 0x6aa69872, blocks: 0x1 00010000:00010000:2.0:1789302899.430927:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789302899.430932:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302899.430942:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.430959:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.430972:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef7200, rc 0) 00000100:00100000:2.0:1789302899.431027:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350583424:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 188us (1057us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.431033:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 482 00000100:00100000:2.0:1789302899.431054:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 483 00000100:00100000:2.0:1789302899.431061:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+21:2347:x1876219350583552:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302899.431072:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302899.431082:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a156a expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:2.0:1789302899.431095:0:6697:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x21:0x0] initial LVB size: 158, mtime: 0x6aa69871, atime: 0x0, ctime: 0x6aa69871, blocks: 0x1 00010000:00010000:2.0:1789302899.431104:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=736s v=(0 0) 00010000:00010000:2.0:1789302899.431107:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302899.431112:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.431122:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 22 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.431131:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f07ef6600, rc 0) 00000100:00100000:2.0:1789302899.431151:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350583552:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 90us (1041us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.431157:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 483 00000100:00100000:1.0:1789302899.447815:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.447887:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.454902:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.454938:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350583936 00000100:00100000:0.0:1789302899.454949:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350583936 00000100:00100000:0.0:1789302899.455003:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 484 00000100:00100000:0.0:1789302899.455014:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+15:2348:x1876219350583936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.455034:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302899.455056:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1571 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302899.455081:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x21:0x0] initial LVB size: 158, mtime: 0x6aa69872, atime: 0x0, ctime: 0x6aa69872, blocks: 0x1 00010000:00010000:0.0:1789302899.455110:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789302899.455115:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302899.455124:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 16 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.455142:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 16 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.455154:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f25f72000, rc 0) 00000100:00100000:0.0:1789302899.455203:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2348:x1876219350583936:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 191us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.455210:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 484 00000100:00100000:2.0:1789302899.457375:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350583808 00000100:00100000:2.0:1789302899.457388:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350583808 00000100:00100000:2.0:1789302899.457462:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350584192 00000100:00100000:2.0:1789302899.457465:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350584192 00000100:00100000:2.0:1789302899.457495:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1650 00000100:00100000:2.0:1789302899.457520:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350583808:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302899.457614:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350583808:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 111us (9808us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.457621:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1650 00000100:00100000:2.0:1789302899.457639:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1651 00000100:00100000:2.0:1789302899.457646:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33540:x1876219350584192:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302899.457680:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02909500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33540:x1876219350584192:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 34us (9793us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.457686:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1651 00000100:00100000:2.0:1789302899.459811:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.459880:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350584448 00000100:00100000:3.0:1789302899.459889:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350584448 00000100:00100000:3.0:1789302899.459939:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1652 00000100:00100000:3.0:1789302899.459948:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33541:x1876219350584448:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302899.460049:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33541:x1876219350584448:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 101us (242us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.460056:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1652 00000100:00100000:2.0:1789302899.464983:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.465993:0:18429:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350584576 00000100:00100000:3.0:1789302899.466006:0:18429:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350584576 00002000:00100000:3.0:1789302899.466064:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x22:0x0] (0->4095) req@ffff960f3fcdce00 x1876219350584576/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.466075:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdce00 with x1876219350584576 ext(0->4095) 00002000:00010000:3.0:1789302899.466080:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fcdce00 00000100:00100000:3.0:1789302899.466106:0:18429:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 134 00000100:00100000:3.0:1789302899.466116:0:18429:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2346:x1876219350584576:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302899.466293:0:18429:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2346:x1876219350584576:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 178us (1319us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302899.466301:0:18429:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 134 00002000:00100000:3.0:1789302899.466307:0:18429:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_004: refresh rw locks for [0x240000400:0x22:0x0] (0->4095) req@ffff960f3fcdce00 x1876219350584576/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/4536 e 0 to 0 dl 1789302910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.466315:0:18429:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdce00 with x1876219350584576 ext(0->4095) 00002000:00010000:3.0:1789302899.466319:0:18429:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fcdce00 00000100:00100000:2.0:1789302899.470128:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.470165:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.470546:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350584704 00000100:00100000:1.0:1789302899.470556:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350584704 00002000:00100000:1.0:1789302899.470601:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x21:0x0] (0->4095) req@ffff960f3fcdea00 x1876219350584704/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302899.470612:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdea00 with x1876219350584704 ext(0->4095) 00002000:00010000:1.0:1789302899.470617:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fcdea00 00000100:00100000:1.0:1789302899.470650:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350584832 00000100:00100000:1.0:1789302899.470654:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350584832 00002000:00100000:1.0:1789302899.470670:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x21:0x0] (0->4095) req@ffff960f3fcdfb80 x1876219350584832/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302899.470678:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdfb80 with x1876219350584832 ext(0->4095) 00002000:00010000:1.0:1789302899.470681:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fcdfb80 00000100:00100000:1.0:1789302899.470697:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 135 00000100:00100000:1.0:1789302899.470705:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350584704:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302899.470784:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 136 00000100:00100000:3.0:1789302899.470793:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350584832:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:1.0:1789302899.470880:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2347:x1876219350584704:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 174us (761us total) trans 0 rc 4096/4096 00000100:00100000:1.0:1789302899.470887:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 135 00000100:00100000:3.0:1789302899.470888:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350584832:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 97us (724us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302899.470894:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 136 00002000:00100000:1.0:1789302899.470894:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_003: refresh rw locks for [0x240000400:0x21:0x0] (0->4095) req@ffff960f3fcdea00 x1876219350584704/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/4536 e 0 to 0 dl 1789302910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.470899:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x21:0x0] (0->4095) req@ffff960f3fcdfb80 x1876219350584832/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/4536 e 0 to 0 dl 1789302910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302899.470902:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdea00 with x1876219350584704 ext(0->4095) 00002000:00010000:1.0:1789302899.470906:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3fcdea00 00002000:00100000:3.0:1789302899.470907:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3fcdfb80 with x1876219350584832 ext(0->4095) 00002000:00010000:3.0:1789302899.470909:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3fcdfb80 00000100:00100000:3.0:1789302899.473130:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.477231:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350584960 00000100:00100000:1.0:1789302899.477244:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350584960 00000100:00100000:1.0:1789302899.477305:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1653 00000100:00100000:1.0:1789302899.477314:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+91:33542:x1876219350584960:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.477335:0:5844:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302899.477355:0:5844:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850768 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1578 expref: -99 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.477400:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 3/1,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.477414:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.477425:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.477548:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe430951850768 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.477565:0:5844:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 92 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.477573:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302899.477577:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302899.477581:0:5844:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9e00, rc 0) 00000100:00100000:1.0:1789302899.477645:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350584960:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 332us (4521us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.477652:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1653 00000100:00100000:3.0:1789302899.483552:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.483660:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350585088 00000100:00100000:0.0:1789302899.483672:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350585088 00000100:00100000:0.0:1789302899.483732:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1654 00000100:00100000:0.0:1789302899.483742:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350585088:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302899.483776:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x35:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302899.483876:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33542:x1876219350585088:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 135us (332us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.483883:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1654 00000100:00100000:1.0:1789302899.492457:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.492673:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350585216 00000100:00100000:2.0:1789302899.492682:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350585216 00000100:00100000:2.0:1789302899.492738:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1655 00000100:00100000:2.0:1789302899.492748:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+92:33540:x1876219350585216:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302899.492769:0:5845:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302899.492793:0:5845:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850776 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a157f expref: -99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.492829:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 3/1,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.492842:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.492854:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.492950:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7c00/0x64fe430951850776 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 94 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.492971:0:5845:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 93 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.492980:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:2.0:1789302899.492983:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302899.492986:0:5845:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517c00, rc 0) 00000100:00100000:2.0:1789302899.493052:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe30700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+93:33540:x1876219350585216:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 306us (604us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.493059:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1655 00000100:00100000:3.0:1789302899.493357:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.493434:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350585344 00000100:00100000:1.0:1789302899.493448:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350585344 00000100:00100000:1.0:1789302899.493483:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 485 00000100:00100000:1.0:1789302899.493492:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2348:x1876219350585344:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.493539:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493554:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302899.493559:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302899.493565:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493577:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493601:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493610:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493618:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493626:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302899.493630:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493644:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493658:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.493665:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2ad758c/0x64fe430951850784 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.493705:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2348:x1876219350585344:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 216us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.493713:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 485 00000100:00100000:3.0:1789302899.496710:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.496799:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350585600 00000100:00100000:2.0:1789302899.496809:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350585600 00000100:00100000:2.0:1789302899.496873:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1656 00000100:00100000:2.0:1789302899.496883:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+93:33542:x1876219350585600:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:2.0:1789302899.496958:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.496976:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.496983:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 4/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.497014:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.497040:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 93 pid: 5844 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302899.497053:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371189248:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302899.497135:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61840 going to sleep for 11 seconds 00000100:00100000:3.0:1789302899.501376:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fcde680 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371189248:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302899.501406:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.502305:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.502415:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350585728 00000100:00100000:1.0:1789302899.502423:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350585728 00010000:00010000:1.0:1789302899.502469:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 94 pid: 5844 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.502526:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 477 00000100:00100000:1.0:1789302899.502536:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+94:2345:x1876219350585728:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.502562:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.502569:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 94 pid: 5844 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.502580:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 94 pid: 5844 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.502604:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9e00/0x64fe43095185076f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1578 expref: 94 pid: 5844 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.502621:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302899.502626:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.502636:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.502643:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.502669:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.502726:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+93:2345:x1876219350585728:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 191us (424us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.502734:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 477 00010000:00010000:2.0:1789302899.503470:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503481:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503719:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503727:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503735:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503741:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302899.503745:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503753:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503764:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:2.0:1789302899.503767:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302899.503772:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 2/0,0 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.503779:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe43095185078b lrc: 0/0,0 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302899.503916:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+93:33542:x1876219350585600:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 7033us (7213us total) trans 4294968376 rc 0/0 00000100:00100000:2.0:1789302899.503925:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1656 00000100:00100000:0.0:1789302899.507676:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.507725:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.507780:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350585856 00000100:00100000:1.0:1789302899.507790:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350585856 00000100:00100000:1.0:1789302899.507848:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350585984 00000100:00100000:1.0:1789302899.507851:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350585984 00000100:00100000:1.0:1789302899.507873:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1657 00000100:00100000:1.0:1789302899.507882:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+94:33541:x1876219350585856:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.507900:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302899.507922:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850792 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1586 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.507956:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.507971:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.507982:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.508078:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8c00/0x64fe430951850792 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.508095:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 95 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.508103:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:1.0:1789302899.508106:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302899.508110:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9800, rc 0) 00000100:00100000:1.0:1789302899.508162:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70dc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+95:33541:x1876219350585856:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 281us (495us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.508170:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1657 00000100:00100000:1.0:1789302899.508187:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1658 00000100:00100000:1.0:1789302899.508194:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+94:33540:x1876219350585984:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302899.508220:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x33:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302899.508286:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+94:33540:x1876219350585984:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 92us (561us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.508293:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1658 00000100:00100000:1.0:1789302899.514141:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.514460:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350586240 00000100:00100000:3.0:1789302899.514470:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350586240 00000100:00100000:3.0:1789302899.514546:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 486 00000100:00100000:3.0:1789302899.514556:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2346:x1876219350586240:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:3.0:1789302899.514599:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514620:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302899.514624:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302899.514630:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514642:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514672:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514681:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514689:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514697:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302899.514702:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514711:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514726:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.514734:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000004d03796/0x64fe4309518507a0 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.514798:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2346:x1876219350586240:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 243us (665us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.514805:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 486 00000100:00100000:1.0:1789302899.521706:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.521748:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.521770:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350586496 00000100:00100000:1.0:1789302899.521778:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350586496 00000100:00100000:1.0:1789302899.521821:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:1.0:1789302899.521829:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22333c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+94:33542:x1876219350586496:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:1.0:1789302899.522026:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22333c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+95:33542:x1876219350586496:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 196us (276us total) trans 4294968377 rc 0/0 00000100:00100000:1.0:1789302899.522033:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 341 00000100:00100000:0.0:1789302899.522195:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350586112 00000100:00100000:0.0:1789302899.522200:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350586112 00000100:00100000:0.0:1789302899.522231:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1659 00000100:00100000:0.0:1789302899.522238:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+95:33541:x1876219350586112:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302899.522270:0:6919:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x34:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302899.522330:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+95:33541:x1876219350586112:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 92us (631us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.522337:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1659 00000100:00100000:3.0:1789302899.523247:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.523380:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350586752 00000100:00100000:0.0:1789302899.523386:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350586752 00000100:00100000:0.0:1789302899.523425:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1660 00000100:00100000:0.0:1789302899.523433:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+95:33540:x1876219350586752:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.523496:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.523535:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 95 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.523545:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.523578:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 95 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.523593:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 95 pid: 5845 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.523614:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371189376:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302899.523674:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302899.525863:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.525939:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350586880 00000100:00100000:1.0:1789302899.525948:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350586880 00010000:00010000:1.0:1789302899.525983:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 96 pid: 5845 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.526033:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 478 00000100:00100000:1.0:1789302899.526043:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+96:2345:x1876219350586880:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.526055:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.526062:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 96 pid: 5845 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.526074:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 96 pid: 5845 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.526095:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:1.0:1789302899.526101:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.526111:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.526118:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.526136:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.526181:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+96:2345:x1876219350586880:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 141us (325us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.526188:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 478 00010000:00010000:0.0:1789302899.529313:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe43095185077d lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a157f expref: 95 pid: 5845 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.529345:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025880 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371189376:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302899.529389:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.529409:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.529416:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.530939:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.530955:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.530962:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.530968:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302899.530974:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.530986:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531003:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00000100:00100000:1.0:1789302899.531009:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:0.0:1789302899.531010:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302899.531018:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531025:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518507a7 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.531210:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbdf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+95:33540:x1876219350586752:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 7777us (7966us total) trans 4294968378 rc 0/0 00000100:00100000:0.0:1789302899.531220:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1660 00000100:00100000:0.0:1789302899.531274:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350587008 00000100:00100000:0.0:1789302899.531282:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350587008 00000100:00100000:0.0:1789302899.531325:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00100000:0.0:1789302899.531334:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350587008:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.531385:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 3/1,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531399:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302899.531403:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302899.531409:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531423:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531450:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531458:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531466:0:6695:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531473:0:6695:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72600 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302899.531477:0:6695:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531484:0:6695:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531495:0:6695:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.531523:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe4309518507ae lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.531558:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2347:x1876219350587008:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 225us (557us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.531564:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00100000:3.0:1789302899.534954:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.535041:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350587264 00000100:00100000:2.0:1789302899.535051:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350587264 00000100:00100000:2.0:1789302899.535109:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 342 00000100:00100000:2.0:1789302899.535119:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+95:33540:x1876219350587264:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302899.535345:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+96:33540:x1876219350587264:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 227us (401us total) trans 4294968379 rc 0/0 00000100:00100000:2.0:1789302899.535353:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 342 00000100:00100000:0.0:1789302899.535791:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.535846:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350587520 00000100:00100000:3.0:1789302899.535851:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350587520 00000100:00100000:3.0:1789302899.535892:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1661 00000100:00100000:3.0:1789302899.535940:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+96:33541:x1876219350587520:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302899.535994:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.536012:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.536022:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.536055:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 96 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.536070:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 96 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.536084:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371189504:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302899.536149:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00aa80 going to sleep for 11 seconds 00000100:00100000:0.0:1789302899.543216:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.543640:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350587648 00000100:00100000:1.0:1789302899.543650:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350587648 00010000:00010000:1.0:1789302899.543690:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 97 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.543778:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 479 00000100:00100000:1.0:1789302899.543789:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:2345:x1876219350587648:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.543804:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.543812:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 97 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.543827:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 97 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.543849:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:1.0:1789302899.543854:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.543864:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.543872:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT flags: 0x50290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.543891:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.543943:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:2345:x1876219350587648:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 157us (736us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.543951:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 479 00010000:00010000:3.0:1789302899.544812:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850799 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1586 expref: 96 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.544832:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbce00 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371189504:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302899.544854:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.544863:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.544868:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.545065:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.545073:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.545079:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.545084:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302899.545088:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.545098:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.545112:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:3.0:1789302899.545115:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:3.0:1789302899.545119:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.545124:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507b5 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.545274:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+96:33541:x1876219350587520:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 9374us (9482us total) trans 4294968380 rc 0/0 00000100:00100000:3.0:1789302899.545283:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1661 00000040:00080000:1.0:1789302899.546093:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302899.546099:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302899.546139:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 13 00000040:00080000:1.0:1789302899.546145:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302899.547347:0:11693:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52d80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371189632:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302899.547454:0:11693:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75780 going to sleep for 11 seconds 00000100:00100000:1.0:1789302899.549443:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.549532:0:5847:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350587904 00000100:00100000:0.0:1789302899.549541:0:5847:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350587904 00000100:00100000:0.0:1789302899.549593:0:5847:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 343 00000100:00100000:0.0:1789302899.549603:0:5847:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:33541:x1876219350587904:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:0.0:1789302899.549820:0:5847:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef30380 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+98:33541:x1876219350587904:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 217us (394us total) trans 4294968381 rc 0/0 00000100:00100000:0.0:1789302899.549827:0:5847:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 343 00000100:00100000:1.0:1789302899.557898:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.557927:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350588032 00000100:00100000:3.0:1789302899.558142:0:11693:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52d80 pname:cluuid:pid:xid:nid:opc:job hsm_cdtr:lustre-MDT0000_UUID:11693:1876219371189632:192.168.203.11@tcp:107:hsm_cdtr.0 00000100:00100000:1.0:1789302899.558326:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350588032 00000100:00100000:1.0:1789302899.558395:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1662 00000100:00100000:1.0:1789302899.558640:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:33543:x1876219350588032:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:1.0:1789302899.558671:0:18427:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x36:0x0] from 18446744073709551615 #0 00000100:00100000:1.0:1789302899.558771:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:33543:x1876219350588032:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 133us (880us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.560607:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.560639:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350588160 00000100:00100000:2.0:1789302899.560647:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350588160 00000100:00100000:1.0:1789302899.564278:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1662 00000100:00100000:2.0:1789302899.564389:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1663 00000100:00100000:2.0:1789302899.564409:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:33544:x1876219350588160:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:2.0:1789302899.564667:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x37:0x0] from 18446744073709551615 #0 00000100:00100000:2.0:1789302899.564764:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:33544:x1876219350588160:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 365us (4160us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.564771:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1663 00000100:00100000:2.0:1789302899.572926:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.573001:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350588416 00000100:00100000:3.0:1789302899.573011:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350588416 00000100:00100000:3.0:1789302899.573077:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1664 00000100:00100000:3.0:1789302899.573086:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+97:33544:x1876219350588416:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302899.573109:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302899.573130:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507bc lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a158d expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.573227:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518507c3 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.573242:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518507c3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.573256:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518507c3 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:3.0:1789302899.573267:0:5843:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x37:0x0]: returning size 158 00010000:00010000:3.0:1789302899.573397:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b800/0x64fe4309518507bc lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a158d expref: 100 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.573415:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518507c3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a158d expref: 99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.573424:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:3.0:1789302899.573427:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1789302899.573431:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00000100:00100000:3.0:1789302899.573529:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+99:33544:x1876219350588416:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 416us (583us total) trans 4294968382 rc 0/0 00000100:00100000:3.0:1789302899.573536:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1664 00000100:00100000:3.0:1789302899.581653:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.581717:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.581840:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350588544 00000100:00100000:1.0:1789302899.581852:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350588544 00000100:00100000:1.0:1789302899.581914:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:1.0:1789302899.581923:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+22:2348:x1876219350588544:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.581969:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.581989:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 22 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.581997:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x240000400:0x23:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302899.582001:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.582006:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 4/1,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.582039:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 22 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.582053:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 22 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.582067:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371189760:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302899.582149:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f25f75780 going to sleep for 11 seconds 00000100:00100000:2.0:1789302899.582357:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350588672 00000100:00100000:2.0:1789302899.582371:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350588672 00000100:00100000:2.0:1789302899.582414:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 79 00000100:00100000:2.0:1789302899.582422:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd07850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+99:2348:x1876219350588672:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302899.582433:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302899.582452:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe4309518507c3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a158d expref: 99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.582470:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:2.0:1789302899.582474:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302899.582528:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd07850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+98:2348:x1876219350588672:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 107us (810us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.582535:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 79 00000100:00100000:3.0:1789302899.584843:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.587742:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.587830:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c92300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:6697:1876219371189760:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302899.587856:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.587919:0:6702:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350589056 00000100:00100000:3.0:1789302899.587926:0:6702:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350589056 00002000:00100000:3.0:1789302899.587976:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x23:0x0] (0->157) req@ffff960f3f026a00 x1876219350589056/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.587986:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f026a00 with x1876219350589056 ext(0->157) 00010000:00010000:3.0:1789302899.587993:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 23 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.588013:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 23 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302899.588022:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3f026a00 00000100:00100000:3.0:1789302899.588056:0:6702:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 21 00000100:00100000:3.0:1789302899.588066:0:6702:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:2347:x1876219350589056:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:3.0:1789302899.588733:0:6702:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:2347:x1876219350589056:12345-192.168.203.11@tcp:4:cp.0 Request processed in 665us (995us total) trans 4294967389 rc 0/0 00000100:00100000:3.0:1789302899.588741:0:6702:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 21 00002000:00100000:3.0:1789302899.588748:0:6702:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_002: refresh rw locks for [0x240000400:0x23:0x0] (0->157) req@ffff960f3f026a00 x1876219350589056/t4294967389(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/448 e 0 to 0 dl 1789302910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.588756:0:6702:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f026a00 with x1876219350589056 ext(0->157) 00010000:00010000:3.0:1789302899.588762:0:6702:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 25 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.588774:0:6702:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 25 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302899.588781:0:6702:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 1 locks timeout for req ffff960f3f026a00 00000100:00100000:1.0:1789302899.592957:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350588928 00000100:00100000:1.0:1789302899.592975:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350588928 00000100:00100000:0.0:1789302899.593126:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1665 00000100:00100000:0.0:1789302899.593137:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+98:33543:x1876219350588928:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.593163:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302899.593187:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f73c00/0x64fe4309518507d8 lrc: 2/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1594 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.593292:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe4309518507df lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.593306:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe4309518507df lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.593318:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe4309518507df lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00200000:0.0:1789302899.593330:0:18427:0:(mdt_handler.c:909:mdt_pack_attr2body()) [0x200000402:0x36:0x0]: returning size 158 00010000:00010000:0.0:1789302899.593479:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f73c00/0x64fe4309518507d8 lrc: 0/0,0 mode: --/CR res: [0x200000002:0x2:0x0].0x0 bits 0x1/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1594 expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.593539:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe4309518507df lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1594 expref: 100 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.593548:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:0.0:1789302899.593551:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1789302899.593554:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f25f72600, rc 0) 00000100:00100000:0.0:1789302899.593603:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbca80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33543:x1876219350588928:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 468us (8767us total) trans 4294968383 rc 0/0 00000100:00100000:0.0:1789302899.593610:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1665 00000100:00100000:3.0:1789302899.605208:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.609935:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350589184 00000100:00100000:1.0:1789302899.609952:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350589184 00000100:00100000:0.0:1789302899.610188:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:0.0:1789302899.610199:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f08599850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+16:2346:x1876219350589184:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.610246:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.610268:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x40000000020000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.610277:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 1 to [0x280000400:0x22:0x0].0x0 since=1s v=(0 0) 00010000:00010000:0.0:1789302899.610281:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302899.610286:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 4/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.610319:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000000020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 16 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.610334:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.610360:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350589440 00000100:00100000:3.0:1789302899.610372:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350589440 00000100:00100000:3.0:1789302899.610405:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 80 00000100:00100000:3.0:1789302899.610413:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:2348:x1876219350589440:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302899.610423:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302899.610448:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe4309518507df lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1594 expref: 100 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.610467:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:3.0:1789302899.610472:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302899.610543:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x70000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 16 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.610550:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+99:2348:x1876219350589440:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 138us (216us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.610556:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 80 00000100:00100000:0.0:1789302899.610559:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371189888:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302899.610609:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2540 going to sleep for 11 seconds 00000100:00100000:3.0:1789302899.611967:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.613868:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350589312 00000100:00100000:0.0:1789302899.613890:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350589312 00010000:00010000:0.0:1789302899.613932:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 25 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.613980:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 480 00000100:00100000:0.0:1789302899.613989:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+25:2348:x1876219350589312:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302899.614002:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302899.614164:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 4/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 25 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614184:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 25 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614207:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe4309518506ab lrc: 0/0,0 mode: --/PW res: [0x240000400:0x23:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a14f3 expref: 25 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614224:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302899.614228:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(1 0) 00010000:00010000:0.0:1789302899.614234:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614244:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 4/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614272:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302899.614322:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2348:x1876219350589312:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 334us (2885us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.614330:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 480 00010000:00010000:0.0:1789302899.614377:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614387:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614421:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614431:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614666:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614676:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302899.614681:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614691:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614703:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.614710:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002c3fd20a/0x64fe4309518507d1 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.614760:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:2348:x1876219350588544:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 32838us (33118us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.614768:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:3.0:1789302899.636182:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.637833:0:18375:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350589568 00000100:00100000:1.0:1789302899.637846:0:18375:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350589568 00002000:00100000:1.0:1789302899.637910:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x22:0x0] (0->157) req@ffff960f3ffbf800 x1876219350589568/t0(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302899.637930:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffbf800 with x1876219350589568 ext(0->157) 00010000:00010000:1.0:1789302899.637939:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 17 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.637957:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 17 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302899.637966:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ffbf800 00000100:00100000:1.0:1789302899.638006:0:18375:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 22 00000100:00100000:1.0:1789302899.638017:0:18375:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2347:x1876219350589568:12345-192.168.203.11@tcp:4:cp.0 00000100:00100000:1.0:1789302899.638549:0:18375:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2347:x1876219350589568:12345-192.168.203.11@tcp:4:cp.0 Request processed in 532us (2372us total) trans 4294967386 rc 0/0 00000100:00100000:1.0:1789302899.638558:0:18375:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 22 00002000:00100000:1.0:1789302899.638565:0:18375:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_003: refresh rw locks for [0x280000400:0x22:0x0] (0->157) req@ffff960f3ffbf800 x1876219350589568/t4294967386(0) o4->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 648/448 e 0 to 0 dl 1789302910 ref 1 fl Complete:H/600/0 rc 0/0 job:'cp.0' uid:0 gid:0 projid:0 00002000:00100000:1.0:1789302899.638574:0:18375:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffbf800 with x1876219350589568 ext(0->157) 00010000:00010000:1.0:1789302899.638580:0:18375:0:(ldlm_extent.c:791:ldlm_lock_prolong_one()) ### refreshed to 100s. ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 19 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.638591:0:18375:0:(ldlm_lockd.c:607:ldlm_refresh_waiting_lock()) ### refreshed to 100s ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 5/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 19 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:1.0:1789302899.638597:0:18375:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 1 locks timeout for req ffff960f3ffbf800 00000100:00100000:1.0:1789302899.640261:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371189888:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302899.640290:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302899.658455:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.658591:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.658750:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350589824 00000100:00100000:3.0:1789302899.658760:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350589824 00000100:00100000:3.0:1789302899.658817:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1666 00000100:00100000:3.0:1789302899.658829:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+99:33544:x1876219350589824:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:3.0:1789302899.658946:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdf480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+99:33544:x1876219350589824:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 118us (499us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.658953:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1666 00000100:00100000:3.0:1789302899.661936:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.661979:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350590080 00000100:00100000:3.0:1789302899.661987:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350590080 00000100:00100000:3.0:1789302899.662046:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 490 00000100:00100000:3.0:1789302899.662055:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+23:2348:x1876219350590080:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:3.0:1789302899.662072:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302899.662091:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518507f4 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a159b expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:3.0:1789302899.662118:0:6695:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x240000400:0x23:0x0] initial LVB size: 158, mtime: 0x6aa69872, atime: 0x0, ctime: 0x6aa69872, blocks: 0x1 00010000:00010000:3.0:1789302899.662135:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=736s v=(0 0) 00010000:00010000:3.0:1789302899.662139:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302899.662148:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518507f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a159b expref: 24 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.662165:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000066c20e73/0x64fe4309518507f4 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a159b expref: 24 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.662177:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345b800, rc 0) 00000100:00100000:3.0:1789302899.662219:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2348:x1876219350590080:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 166us (292us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.662226:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 490 00000100:00100000:1.0:1789302899.662712:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350589952 00000100:00100000:1.0:1789302899.662717:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350589952 00010000:00010000:1.0:1789302899.662743:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 19 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.662779:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 481 00000100:00100000:1.0:1789302899.662786:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350589952:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.662795:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.662812:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 4/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x60000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 19 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.662824:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x50000400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 19 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.662841:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000994d4816/0x64fe4309518505af lrc: 0/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->4095) gid 0 flags: 0x44801400020020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1460 expref: 19 pid: 6697 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.662855:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.662858:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(1 0) 00010000:00010000:1.0:1789302899.662862:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.662883:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 4/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.662907:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.662933:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350589952:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 147us (4343us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.662938:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 481 00010000:00010000:1.0:1789302899.662960:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.662969:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.662994:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.663002:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.663010:0:13439:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.663023:0:13439:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f73c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302899.663028:0:13439:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.663039:0:13439:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.663055:0:13439:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.663062:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507ed lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.663101:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f08599850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2346:x1876219350589184:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 52903us (57902us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.663108:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:3.0:1789302899.668877:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.668920:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.669051:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350590336 00000100:00100000:2.0:1789302899.669061:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350590336 00000100:00100000:2.0:1789302899.669114:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350590592 00000100:00100000:2.0:1789302899.669117:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350590592 00000100:00100000:2.0:1789302899.669138:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1667 00000100:00100000:2.0:1789302899.669147:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33543:x1876219350590336:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302899.669244:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbfb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33543:x1876219350590336:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 98us (388us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.669251:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1667 00000100:00100000:2.0:1789302899.669269:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1668 00000100:00100000:2.0:1789302899.669275:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+99:33544:x1876219350590592:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302899.669307:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+99:33544:x1876219350590592:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 32us (386us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.669314:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1668 00000100:00100000:0.0:1789302899.694651:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.694972:0:6701:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350590720 00000100:00100000:3.0:1789302899.694982:0:6701:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350590720 00002000:00100000:3.0:1789302899.695039:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x23:0x0] (0->4095) req@ffff960f3f024000 x1876219350590720/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.695050:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f024000 with x1876219350590720 ext(0->4095) 00002000:00010000:3.0:1789302899.695056:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f024000 00000100:00100000:3.0:1789302899.695085:0:6701:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 137 00000100:00100000:3.0:1789302899.695095:0:6701:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350590720:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:3.0:1789302899.695277:0:6701:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350590720:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 183us (634us total) trans 0 rc 4096/4096 00000100:00100000:3.0:1789302899.695285:0:6701:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 137 00002000:00100000:3.0:1789302899.695291:0:6701:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0000 ll_ost_io00_001: refresh rw locks for [0x240000400:0x23:0x0] (0->4095) req@ffff960f3f024000 x1876219350590720/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/4536 e 0 to 0 dl 1789302910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:3.0:1789302899.695299:0:6701:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3f024000 with x1876219350590720 ext(0->4095) 00002000:00010000:3.0:1789302899.695302:0:6701:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0000: refreshed 0 locks timeout for req ffff960f3f024000 00000100:00100000:0.0:1789302899.721552:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.721622:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350590848 00000100:00100000:0.0:1789302899.721631:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350590848 00000100:00100000:0.0:1789302899.721683:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00100000:0.0:1789302899.721693:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2348:x1876219350590848:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.721709:0:18578:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302899.721729:0:18578:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15a2 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00010000:0.0:1789302899.721756:0:18578:0:(ofd_lvb.c:151:ofd_lvbo_init()) res: [0x280000400:0x22:0x0] initial LVB size: 158, mtime: 0x6aa69872, atime: 0x0, ctime: 0x6aa69872, blocks: 0x1 00010000:00010000:0.0:1789302899.721775:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=736s v=(0 0) 00010000:00010000:0.0:1789302899.721779:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302899.721788:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.721804:0:18578:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 18 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.721815:0:18578:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f25f73c00, rc 0) 00000100:00100000:0.0:1789302899.721861:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350590848:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 171us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.721867:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00100000:3.0:1789302899.739009:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.739051:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.740771:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350590976 00000100:00100000:2.0:1789302899.740781:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350590976 00000100:00100000:2.0:1789302899.740828:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350591232 00000100:00100000:2.0:1789302899.740832:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350591232 00000100:00100000:2.0:1789302899.740858:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1669 00000100:00100000:2.0:1789302899.740950:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33544:x1876219350590976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:2.0:1789302899.740971:0:5843:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302899.740992:0:5843:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850802 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15a9 expref: -99 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.741037:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 3/1,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.741051:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.741061:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.741154:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850802 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 102 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.741171:0:5843:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 101 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.741186:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:2.0:1789302899.741189:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:2.0:1789302899.741193:0:5843:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516200, rc 0) 00000100:00100000:2.0:1789302899.741245:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:33544:x1876219350590976:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 299us (2244us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.741251:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1669 00000100:00100000:2.0:1789302899.741268:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1670 00000100:00100000:2.0:1789302899.741275:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33543:x1876219350591232:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00000100:00100000:2.0:1789302899.741315:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33543:x1876219350591232:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 41us (2264us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.741322:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1670 00000100:00100000:3.0:1789302899.749096:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371190016:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302899.749166:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302899.749335:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371190016 00000100:00100000:3.0:1789302899.749342:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371190016 00000100:00100000:3.0:1789302899.749401:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 223 00000100:00100000:3.0:1789302899.749410:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371190016:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302899.749442:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 107151360 pending 8519680 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302899.749486:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371190016:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.749493:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 223 00000100:00100000:3.0:1789302899.749562:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.749601:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371190016:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302899.762807:0:6700:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350591360 00000100:00100000:0.0:1789302899.762823:0:6700:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350591360 00002000:00100000:0.0:1789302899.762889:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x22:0x0] (0->4095) req@ffff960f3ffbd180 x1876219350591360/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/0 e 0 to 0 dl 1789302910 ref 1 fl New:/600/ffffffff rc 0/-1 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302899.762901:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffbd180 with x1876219350591360 ext(0->4095) 00002000:00010000:0.0:1789302899.762908:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ffbd180 00000100:00100000:0.0:1789302899.762944:0:6700:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 138 00000100:00100000:0.0:1789302899.762955:0:6700:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2345:x1876219350591360:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 00000100:00100000:0.0:1789302899.763167:0:6700:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_io00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2345:x1876219350591360:12345-192.168.203.11@tcp:3:lhsmtool_posix.0 Request processed in 213us (13604us total) trans 0 rc 4096/4096 00000100:00100000:0.0:1789302899.763175:0:6700:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 138 00002000:00100000:0.0:1789302899.763182:0:6700:0:(ofd_dev.c:2711:ofd_rw_hpreq_check()) @@@ lustre-OST0001 ll_ost_io00_000: refresh rw locks for [0x280000400:0x22:0x0] (0->4095) req@ffff960f3ffbd180 x1876219350591360/t0(0) o3->584f7d4c-528a-4a63-ad91-f53185ae8e19@192.168.203.11@tcp:475/0 lens 488/4536 e 0 to 0 dl 1789302910 ref 1 fl Complete:/600/0 rc 4096/4096 job:'lhsmtool_posix.0' uid:0 gid:0 projid:0 00002000:00100000:0.0:1789302899.763191:0:6700:0:(ofd_dev.c:2567:ofd_prolong_extent_locks()) Prolong locks for req ffff960f3ffbd180 with x1876219350591360 ext(0->4095) 00002000:00010000:0.0:1789302899.763195:0:6700:0:(ofd_dev.c:2716:ofd_rw_hpreq_check()) lustre-OST0001: refreshed 0 locks timeout for req ffff960f3ffbd180 00000100:00100000:3.0:1789302899.782357:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.782461:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350591488 00000100:00100000:0.0:1789302899.782472:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350591488 00000100:00100000:0.0:1789302899.782560:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1671 00000100:00100000:0.0:1789302899.782571:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33544:x1876219350591488:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302899.782603:0:5845:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x37:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302899.782703:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33544:x1876219350591488:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 134us (353us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.782711:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1671 00000100:00100000:3.0:1789302899.789144:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.805695:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.805912:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350591616 00000100:00100000:0.0:1789302899.805923:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350591616 00000100:00100000:0.0:1789302899.805962:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350591744 00000100:00100000:0.0:1789302899.805966:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350591744 00000100:00100000:0.0:1789302899.806024:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 492 00000100:00100000:0.0:1789302899.806035:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350591744:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.806082:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 3/1,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806111:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302899.806115:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302899.806122:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806137:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806174:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 3/1,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806183:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806190:0:6697:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806197:0:6697:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302899.806201:0:6697:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806209:0:6697:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806223:0:6697:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.806231:0:6697:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000094d19f9f/0x64fe430951850810 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.806303:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350591744:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 270us (617us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.806310:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 492 00000100:00100000:1.0:1789302899.807170:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1672 00000100:00100000:1.0:1789302899.807181:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:33543:x1876219350591616:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 00010000:00010000:1.0:1789302899.807197:0:18427:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302899.807214:0:18427:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe430951850817 lrc: 2/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15b0 expref: -99 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.807246:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 3/1,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.807260:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.807268:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 3/1,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.807356:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8e00/0x64fe430951850817 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 102 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.807388:0:18427:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.807397:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:1.0:1789302899.807410:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:1.0:1789302899.807414:0:18427:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc 0) 00000100:00100000:1.0:1789302899.807460:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:33543:x1876219350591616:12345-192.168.203.11@tcp:101:lhsmtool_posix.0 Request processed in 282us (18326us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.807467:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1672 00000100:00100000:2.0:1789302899.827313:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.827470:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.847102:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350591872 00000100:00100000:0.0:1789302899.847118:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350591872 00000100:00100000:0.0:1789302899.848726:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350592128 00000100:00100000:0.0:1789302899.848738:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350592128 00000100:00100000:0.0:1789302899.848771:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1673 00000100:00100000:0.0:1789302899.848779:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:33543:x1876219350591872:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 00000004:00000080:0.0:1789302899.848822:0:5843:0:(mdt_handler.c:7972:mdt_fid2path()) path get [0x200000402:0x36:0x0] from 18446744073709551615 #0 00000100:00100000:0.0:1789302899.848943:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:33543:x1876219350591872:12345-192.168.203.11@tcp:53:lhsmtool_posix.0 Request processed in 163us (21637us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.848952:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1673 00000100:00100000:0.0:1789302899.848969:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1674 00000100:00100000:0.0:1789302899.848976:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:33544:x1876219350592128:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:0.0:1789302899.849028:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.849045:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 101 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.849053:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.849082:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 101 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.849096:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 101 pid: 5843 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.849110:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371190144:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302899.849175:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2d80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302899.866449:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371190144:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302899.866482:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.882391:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.882554:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302899.883609:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350592256 00000100:00100000:2.0:1789302899.883620:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350592256 00000100:00100000:2.0:1789302899.883673:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 493 00000100:00100000:2.0:1789302899.883682:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350592256:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 00010000:00010000:2.0:1789302899.883721:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 3/1,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883737:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302899.883741:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302899.883747:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883760:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883789:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 3/1,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883797:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883806:0:6696:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883813:0:6696:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302899.883817:0:6696:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883825:0:6696:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883838:0:6696:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302899.883846:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006438736a/0x64fe43095185082c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819400000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302899.883984:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2348:x1876219350592256:12345-192.168.203.11@tcp:1:lhsmtool_posix.0 Request processed in 302us (1429us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302899.883991:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 493 00000100:00100000:1.0:1789302899.885306:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350592384 00000100:00100000:1.0:1789302899.885316:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350592384 00010000:00010000:1.0:1789302899.885346:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 102 pid: 5843 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302899.885393:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 482 00000100:00100000:1.0:1789302899.885401:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:2347:x1876219350592384:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302899.885414:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302899.885420:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 102 pid: 5843 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.885795:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 102 pid: 5843 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.885820:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe430951850809 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a9 expref: 102 pid: 5843 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.885834:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:1.0:1789302899.885839:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.885848:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.885854:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302899.885951:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302899.885999:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:2347:x1876219350592384:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 599us (3615us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302899.886007:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 482 00010000:00010000:0.0:1789302899.886394:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886403:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886629:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886637:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886642:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886648:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302899.886651:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886658:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886668:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x37:0x0].0x0 (ffff960f0cdf2600) 00010000:00010000:0.0:1789302899.886671:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302899.886675:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.886680:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850825 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x37:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.886794:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:33544:x1876219350592128:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 37817us (59323us total) trans 4294968384 rc 0/0 00000100:00100000:0.0:1789302899.886806:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1674 00000100:00100000:1.0:1789302899.889434:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.889655:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350592640 00000100:00100000:3.0:1789302899.889664:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350592640 00000100:00100000:3.0:1789302899.889723:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1675 00000100:00100000:3.0:1789302899.889733:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:33543:x1876219350592640:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 00010000:00010000:3.0:1789302899.889801:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.889818:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.889828:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 4/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.889870:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 101 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.889884:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 101 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.889898:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371190272:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302899.889971:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3f00aa80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302899.905467:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.911228:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.913090:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371190272:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302899.913120:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.913299:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350593024 00000100:00100000:3.0:1789302899.913305:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350593024 00010000:00010000:3.0:1789302899.913342:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 102 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302899.913393:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 483 00000100:00100000:3.0:1789302899.913409:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:2348:x1876219350593024:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302899.913423:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302899.913430:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 102 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.913443:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 102 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.913465:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9200/0x64fe43095185081e lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b0 expref: 102 pid: 18427 timeout: 836 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.913480:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:3.0:1789302899.913484:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 5 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.913494:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.913525:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 5 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302899.913547:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302899.913601:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:2348:x1876219350593024:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 199us (2382us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302899.913609:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 483 00010000:00010000:0.0:1789302899.914709:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914720:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914923:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914931:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914937:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914942:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345aa00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302899.914947:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914955:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 4/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914968:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:0.0:1789302899.914971:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 20 00010000:00010000:0.0:1789302899.914975:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 2/0,0 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302899.914984:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345aa00/0x64fe430951850833 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x20/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302899.915699:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:33543:x1876219350592640:12345-192.168.203.11@tcp:57:lhsmtool_posix.0 Request processed in 25961us (26266us total) trans 4294968385 rc 0/0 00000100:00100000:0.0:1789302899.915710:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1675 00000040:00080000:0.0:1789302899.915829:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302899.915833:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000100:00100000:2.0:1789302899.916430:0:5846:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350592896 00000100:00100000:2.0:1789302899.916448:0:5846:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350592896 00000100:00100000:2.0:1789302899.916488:0:5846:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 344 00000100:00100000:2.0:1789302899.916496:0:5846:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:33544:x1876219350592896:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:2.0:1789302899.916663:0:5846:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:33544:x1876219350592896:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 166us (11206us total) trans 4294968386 rc 0/0 00000100:00100000:2.0:1789302899.916669:0:5846:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 344 00000040:00080000:0.0:1789302899.919374:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 13 00000040:00080000:0.0:1789302899.919392:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302899.930543:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302899.930729:0:12976:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350593280 00000100:00100000:3.0:1789302899.930739:0:12976:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350593280 00000100:00100000:3.0:1789302899.930798:0:12976:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 345 00000100:00100000:3.0:1789302899.930808:0:12976:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:33543:x1876219350593280:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 00000100:00100000:3.0:1789302899.931301:0:12976:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c91180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+103:33543:x1876219350593280:12345-192.168.203.11@tcp:35:lhsmtool_posix.0 Request processed in 492us (798us total) trans 4294968387 rc 0/0 00000100:00100000:3.0:1789302899.931313:0:12976:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 345 00000040:00080000:0.0:1789302899.956115:0:22741:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302899.956126:0:22741:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302899.956208:0:22741:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 13 00000040:00080000:0.0:1789302899.956214:0:22741:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302899.994803:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.994845:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.994869:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302899.994888:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302899.994942:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350593664 00000100:00100000:0.0:1789302899.994953:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350593664 00000100:00100000:2.0:1789302899.994986:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350593408 00000100:00100000:2.0:1789302899.994991:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350593408 00000100:00100000:0.0:1789302899.995019:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350593792 00000100:00100000:2.0:1789302899.995020:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 626 00000100:00100000:0.0:1789302899.995022:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350593792 00000100:00100000:2.0:1789302899.995028:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350593408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302899.995041:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 544 00000100:00100000:0.0:1789302899.995046:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350593664:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:1.0:1789302899.995093:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302899.995142:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+16:2346:x1876219350593664:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 94us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302899.995149:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 544 00000100:00100000:0.0:1789302899.995167:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 545 00000100:00100000:0.0:1789302899.995175:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350593792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302900.000716:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350593408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 5682us (5914us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302900.000731:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 626 00000100:00100000:2.0:1789302900.004038:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350593536 00000100:00100000:2.0:1789302900.004049:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350593536 00000100:00080000:2.0:1789302900.004076:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302900.004128:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 520 00000100:00100000:2.0:1789302900.004139:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:2345:x1876219350593536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302900.004212:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c93100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:2345:x1876219350593536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 75us (9365us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302900.004222:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 520 00000100:00100000:1.0:1789302900.008004:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350593792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 12818us (13103us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302900.008032:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 545 00000100:00100000:1.0:1789302900.036132:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302900.036851:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350593920 00000100:00100000:0.0:1789302900.036862:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350593920 00000100:00100000:0.0:1789302900.036927:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 494 00000100:00100000:0.0:1789302900.036938:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350593920:12345-192.168.203.11@tcp:101:rm.0 00010000:00010000:0.0:1789302900.036955:0:6696:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302900.036973:0:6696:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15b7 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.037002:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=3s v=(0 0) 00010000:00010000:0.0:1789302900.037007:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302900.037028:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 18 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.037048:0:6696:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 18 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.037061:0:6696:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f25f72600, rc 0) 00000100:00100000:0.0:1789302900.037119:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2346:x1876219350593920:12345-192.168.203.11@tcp:101:rm.0 Request processed in 184us (996us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302900.037125:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 494 00000100:00080000:1.0:1789302900.037194:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302900.041838:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.041876:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.041904:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.041925:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.041948:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.041970:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.041987:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.042002:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.042026:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302900.042048:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350594048 00000100:00100000:1.0:1789302900.042057:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350594048 00000100:00100000:1.0:1789302900.042097:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350594304 00000100:00100000:1.0:1789302900.042100:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350594304 00000100:00100000:1.0:1789302900.042109:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350594560 00000100:00100000:1.0:1789302900.042111:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350594560 00000100:00100000:1.0:1789302900.042120:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350594816 00000100:00100000:1.0:1789302900.042123:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350594816 00000100:00100000:1.0:1789302900.042132:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350594176 00000100:00100000:1.0:1789302900.042135:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350594176 00000100:00100000:1.0:1789302900.042142:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350594688 00000100:00100000:1.0:1789302900.042144:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350594688 00000100:00100000:1.0:1789302900.042151:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350594432 00000100:00100000:1.0:1789302900.042153:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350594432 00000100:00100000:1.0:1789302900.042161:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350595200 00000100:00100000:1.0:1789302900.042163:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350595200 00000100:00100000:1.0:1789302900.042178:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 495 00000100:00100000:1.0:1789302900.042187:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2347:x1876219350594048:12345-192.168.203.11@tcp:101:ll_agl.0 00010000:00010000:1.0:1789302900.042206:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302900.042226:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15be expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042254:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=3s v=(0 0) 00010000:00010000:1.0:1789302900.042258:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302900.042267:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042286:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042300:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9600, rc 0) 00000100:00100000:1.0:1789302900.042350:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2347:x1876219350594048:12345-192.168.203.11@tcp:101:ll_agl.0 Request processed in 163us (518us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302900.042357:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 495 00000100:00100000:1.0:1789302900.042387:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 496 00000100:00100000:1.0:1789302900.042394:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2347:x1876219350594304:12345-192.168.203.11@tcp:101:ll_agl.0 00010000:00010000:1.0:1789302900.042401:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302900.042409:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15c5 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042422:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=2s v=(0 0) 00010000:00010000:1.0:1789302900.042425:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302900.042436:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042453:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042463:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8c00, rc 0) 00000100:00100000:1.0:1789302900.042484:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2347:x1876219350594304:12345-192.168.203.11@tcp:101:ll_agl.0 Request processed in 90us (607us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302900.042490:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 496 00000100:00100000:1.0:1789302900.042529:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 497 00000100:00100000:1.0:1789302900.042536:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2347:x1876219350594560:12345-192.168.203.11@tcp:101:ll_agl.0 00010000:00010000:1.0:1789302900.042542:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302900.042550:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 2/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15da expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042562:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302900.042565:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302900.042571:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042581:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042589:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9e00, rc 0) 00000100:00100000:3.0:1789302900.042604:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:1.0:1789302900.042608:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2347:x1876219350594560:12345-192.168.203.11@tcp:101:ll_agl.0 Request processed in 74us (704us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302900.042613:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2347:x1876219350594816:12345-192.168.203.11@tcp:101:ll_agl.0 00000100:00100000:1.0:1789302900.042617:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 497 00010000:00010000:3.0:1789302900.042621:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00000100:00100000:1.0:1789302900.042629:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 499 00010000:00010000:3.0:1789302900.042632:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15e8 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302900.042634:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2345:x1876219350594176:12345-192.168.203.11@tcp:101:ll_sa.0 00010000:00010000:1.0:1789302900.042639:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302900.042646:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=1s v=(0 0) 00010000:00010000:1.0:1789302900.042647:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 2/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15cc expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042649:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302900.042655:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 23 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042656:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=2s v=(0 0) 00010000:00010000:1.0:1789302900.042659:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302900.042664:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 23 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042667:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 23 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042674:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 23 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042676:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a200, rc 0) 00010000:00010000:1.0:1789302900.042682:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b9200, rc 0) 00000100:00100000:1.0:1789302900.042695:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+23:2345:x1876219350594176:12345-192.168.203.11@tcp:101:ll_sa.0 Request processed in 60us (724us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302900.042700:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 499 00000100:00100000:3.0:1789302900.042702:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+23:2347:x1876219350594816:12345-192.168.203.11@tcp:101:ll_agl.0 Request processed in 91us (777us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302900.042708:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:1.0:1789302900.042711:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:1.0:1789302900.042717:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2345:x1876219350594688:12345-192.168.203.11@tcp:101:ll_sa.0 00000100:00100000:3.0:1789302900.042718:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00100000:3.0:1789302900.042723:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2345:x1876219350594432:12345-192.168.203.11@tcp:101:ll_sa.0 00010000:00010000:1.0:1789302900.042723:0:6697:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302900.042728:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302900.042730:0:6697:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15e1 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042734:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 2/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15d3 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042746:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302900.042748:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=2s v=(0 0) 00010000:00010000:1.0:1789302900.042749:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302900.042750:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302900.042754:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 23 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042755:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 23 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042766:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 23 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.042766:0:6697:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 23 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042776:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345a600, rc 0) 00010000:00010000:1.0:1789302900.042776:0:6697:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e024b8e00, rc 0) 00000100:00100000:3.0:1789302900.042792:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+23:2345:x1876219350594432:12345-192.168.203.11@tcp:101:ll_sa.0 Request processed in 69us (789us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302900.042798:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00100000:3.0:1789302900.042811:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 502 00000100:00100000:3.0:1789302900.042817:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2345:x1876219350595200:12345-192.168.203.11@tcp:101:ll_sa.0 00010000:00010000:3.0:1789302900.042823:0:6695:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302900.042831:0:6695:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15ef expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.042842:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=1s v=(0 0) 00010000:00010000:3.0:1789302900.042845:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302900.042850:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 20 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.043423:0:6695:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 20 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.043445:0:6695:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345aa00, rc 0) 00000100:00100000:1.0:1789302900.045067:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2345:x1876219350594688:12345-192.168.203.11@tcp:101:ll_sa.0 Request processed in 2340us (3068us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302900.045078:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:3.0:1789302900.045114:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2345:x1876219350595200:12345-192.168.203.11@tcp:101:ll_sa.0 Request processed in 2295us (3085us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302900.045120:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350595072 00000100:00100000:3.0:1789302900.045121:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 502 00000100:00100000:1.0:1789302900.045128:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350595072 00000100:00100000:1.0:1789302900.045222:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1676 00000100:00100000:1.0:1789302900.045231:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:33545:x1876219350595072:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:1.0:1789302900.045286:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1789302900.045312:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe43095184fd19 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0feb expref: 118 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045329:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:1.0:1789302900.045338:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe4309518506f1 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1540 expref: 117 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045347:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302900.045351:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302900.045385:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045408:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045418:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045431:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045439:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045445:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045560:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045572:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 102 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045580:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045612:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 102 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.045625:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 102 pid: 5845 timeout: 837 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302900.045638:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70e300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371190400:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302900.045697:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33ff9600 going to sleep for 11 seconds 00000100:00100000:3.0:1789302900.056564:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70e300 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371190400:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302900.056595:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302900.061685:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302900.062093:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350595328 00000100:00100000:2.0:1789302900.062104:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350595328 00000100:00080000:3.0:1789302900.062164:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302900.062169:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 103 pid: 5845 timeout: 837 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302900.062210:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 484 00000100:00100000:2.0:1789302900.062220:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+103:2347:x1876219350595328:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302900.062234:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302900.062242:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 103 pid: 5845 timeout: 837 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302900.062255:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 103 pid: 5845 timeout: 837 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302900.062287:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf0c00/0x64fe43095184fce8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2e:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a0fd6 expref: 103 pid: 5845 timeout: 837 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302900.062302:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:2.0:1789302900.062307:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302900.062316:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302900.062324:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302900.062366:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302900.062428:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:2347:x1876219350595328:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 209us (753us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302900.062436:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 484 00010000:00010000:3.0:1789302900.062456:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.062465:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302900.063185:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].16, 64 off9288 00000004:00080000:3.0:1789302900.063196:0:5843:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 737073693686ns, next at 738073691349ns, rc = 0 00010000:00010000:3.0:1789302900.063266:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063278:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063288:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063296:0:5843:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063303:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063309:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063328:0:5843:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302900.063349:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063355:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063375:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063382:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302900.063387:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063394:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063414:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302900.063418:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302900.063422:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063428:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8200/0x64fe430951850887 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2e:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063446:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063453:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063459:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063464:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b8000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302900.063467:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063473:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063481:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302900.063484:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302900.063487:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.063492:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8000/0x64fe430951850879 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302900.063527:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000dd887550 reply state 0000000056ed0638 transno 4294968388 ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302900.063537:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302900.063604:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:33545:x1876219350595072:12345-192.168.203.11@tcp:36:rm.0 Request processed in 18372us (21653us total) trans 4294968388 rc 0/0 00000100:00100000:3.0:1789302900.063612:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1676 00010000:00010000:3.0:1789302900.067628:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302900.067643:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.067654:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302900.067659:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.067669:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.067686:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2e:0x0].0x0 (ffff960f0cdf2700) 00010000:00010000:3.0:1789302900.067689:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302900.067694:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.067699:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185088e lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2e:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302900.067717:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302900.067719:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302900.067720:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302900.068993:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.069007:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.069024:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.069034:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e024b9800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302900.069039:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.069045:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.069058:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x156975b9 (ffff960f2c94d200) 00010000:00010000:0.0:1789302900.069061:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302900.069065:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302900.069071:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9800/0x64fe430951850880 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x156975b9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302900.364722:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302900.962732:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302900.962742:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302900.963075:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:2.0:1789302900.963082:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789302901.122124:0:22792:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302901.122136:0:22792:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302901.122227:0:22792:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:3.0:1789302901.122234:0:22792:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302901.222684:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302901.222890:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350595584 00000100:00100000:1.0:1789302901.222904:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350595584 00000100:00100000:1.0:1789302901.222962:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1677 00000100:00100000:1.0:1789302901.222972:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33557:x1876219350595584:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:1.0:1789302901.223024:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302901.223050:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe43095184fdeb lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a103f expref: 117 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223068:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:1.0:1789302901.223073:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302901.223095:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223109:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223119:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223134:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223141:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223153:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223261:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223271:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223279:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223310:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 101 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302901.223335:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 101 pid: 5844 timeout: 838 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302901.223360:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f70c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371190528:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302901.223451:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f33ff9180 going to sleep for 11 seconds 00000100:00100000:1.0:1789302901.227124:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f70c700 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371190528:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302901.227154:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302901.237856:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302901.237907:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350595712 00000100:00100000:2.0:1789302901.237918:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350595712 00010000:00010000:2.0:1789302901.237959:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 102 pid: 5844 timeout: 838 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302901.238006:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 485 00000100:00100000:2.0:1789302901.238018:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd04c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+102:2347:x1876219350595712:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302901.238036:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302901.238043:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 102 pid: 5844 timeout: 838 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302901.238059:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 102 pid: 5844 timeout: 838 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302901.238083:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1200/0x64fe43095184fdba lrc: 0/0,0 mode: --/PR res: [0x200000402:0x2f:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a102a expref: 102 pid: 5844 timeout: 838 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302901.238099:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:2.0:1789302901.238106:0:13636:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302901.238117:0:13636:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302901.238125:0:13636:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302901.238168:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302901.238227:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.238240:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302901.238301:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd04c50 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:2347:x1876219350595712:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 216us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302901.238313:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 485 00000040:00080000:0.0:1789302901.239482:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].19, 64 off9480 00000004:00080000:0.0:1789302901.239494:0:5844:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 738249991496ns, next at 739249988711ns, rc = 0 00010000:00010000:0.0:1789302901.239594:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 3/0,1 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239611:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239627:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239635:0:5844:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239641:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239647:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239664:0:5844:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789302901.239685:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239692:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239698:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239704:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302901.239709:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239716:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239727:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:0.0:1789302901.239730:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302901.239735:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239742:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe4309518508a3 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x2f:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239760:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239767:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239773:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239778:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302901.239781:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239787:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239794:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302901.239798:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302901.239801:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239807:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe430951850895 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:0.0:1789302901.239817:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000944565c9 reply state 000000001f541802 transno 4294968389 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302901.239830:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:0.0:1789302901.239901:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:33557:x1876219350595584:12345-192.168.203.11@tcp:36:rm.0 Request processed in 16927us (17222us total) trans 4294968389 rc 0/0 00000100:00100000:0.0:1789302901.239909:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1677 00010000:00010000:0.0:1789302901.239939:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302901.239945:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239952:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302901.239955:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239961:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239969:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2f:0x0].0x0 (ffff960f0cdf2b00) 00010000:00010000:0.0:1789302901.239971:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302901.239974:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 1/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239979:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508aa lrc: 0/0,0 mode: --/PW res: [0x200000402:0x2f:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.239992:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302901.239994:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302901.239995:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302901.244667:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.244687:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.244696:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.244702:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbf600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302901.244706:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.244714:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.244730:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x3fac6341 (ffff960f2c94d500) 00010000:00010000:0.0:1789302901.244733:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302901.244739:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302901.244745:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbf600/0x64fe43095185089c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x3fac6341 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302901.391696:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302902.413628:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302902.548136:0:22843:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302902.548144:0:22843:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302902.548208:0:22843:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:3.0:1789302902.548215:0:22843:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302902.709329:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.709371:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.709404:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.709433:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596224 00000100:00100000:1.0:1789302902.709444:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596224 00000100:00100000:1.0:1789302902.709524:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 503 00000100:00100000:1.0:1789302902.709535:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2345:x1876219350596224:12345-192.168.203.11@tcp:101:ll_agl.0 00010000:00010000:1.0:1789302902.709553:0:8571:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302902.709571:0:8571:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe4309518508b1 lrc: 2/0,0 mode: --/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15fd expref: -99 pid: 8571 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.709596:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=3s v=(0 0) 00010000:00010000:1.0:1789302902.709601:0:8571:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x23:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302902.709610:0:8571:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe4309518508b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15fd expref: 21 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.709640:0:8571:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000da7bf650/0x64fe4309518508b1 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x23:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15fd expref: 21 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.709654:0:8571:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3fcbf600, rc 0) 00000100:00100000:1.0:1789302902.709705:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2345:x1876219350596224:12345-192.168.203.11@tcp:101:ll_agl.0 Request processed in 172us (300us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302902.709712:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 503 00000100:00080000:1.0:1789302902.709733:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302902.709881:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596096 00000100:00100000:0.0:1789302902.709886:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596096 00000100:00100000:0.0:1789302902.709917:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 89 00000100:00100000:0.0:1789302902.709925:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:2348:x1876219350596096:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302902.709950:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302902.709964:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508b8 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a15f6 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710012:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710030:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710040:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710056:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710064:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710071:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710121:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710129:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710136:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710142:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302902.710147:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710155:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710167:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302902.710170:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302902.710175:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710181:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe4309518508bf lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710192:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710199:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710205:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710211:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f121dca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302902.710214:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710220:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710227:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xbb834312 (ffff960f0cf83400) 00010000:00010000:0.0:1789302902.710230:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302902.710233:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710239:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508c6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710265:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508b8 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15f6 expref: 118 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710276:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302902.710279:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302902.710282:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe4309518508b8 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15f6 expref: 118 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.710291:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f25f72400, rc -2) 00000100:00100000:0.0:1789302902.710336:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:2348:x1876219350596096:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 411us (964us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302902.710356:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 89 00000100:00100000:2.0:1789302902.710462:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350595968 00000100:00100000:2.0:1789302902.710467:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350595968 00000100:00100000:2.0:1789302902.710496:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1678 00000100:00100000:2.0:1789302902.710521:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+117:33567:x1876219350595968:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:2.0:1789302902.710548:0:18427:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302902.710562:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe43095184febd lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1093 expref: 117 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710574:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:2.0:1789302902.710578:0:18427:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302902.710589:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710598:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710606:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710630:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710639:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710646:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710711:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710722:0:18427:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 100 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710730:0:18427:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 4/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710762:0:18427:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 100 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.710776:0:18427:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 100 pid: 5843 timeout: 839 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302902.710790:0:18427:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371190656:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302902.710848:0:18427:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0cf61180 going to sleep for 11 seconds 00000100:00080000:1.0:1789302902.712441:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302902.717230:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302902.717419:0:18427:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000_UUID:18427:1876219371190656:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302902.717449:0:18427:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.726811:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596352 00000100:00100000:1.0:1789302902.726830:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596352 00010000:00010000:1.0:1789302902.726907:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 101 pid: 5843 timeout: 839 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.726950:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 486 00000100:00100000:1.0:1789302902.726961:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+101:2347:x1876219350596352:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302902.726977:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302902.726984:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 101 pid: 5843 timeout: 839 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.726997:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 101 pid: 5843 timeout: 839 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727023:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f800/0x64fe43095184fe8c lrc: 0/0,0 mode: --/PR res: [0x200000402:0x30:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a107e expref: 101 pid: 5843 timeout: 839 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727042:0:5829:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:1.0:1789302902.727047:0:5829:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,1 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727056:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727065:0:5829:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727092:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302902.727152:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+100:2347:x1876219350596352:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 193us (9933us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302902.727159:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 486 00010000:00010000:1.0:1789302902.727198:0:18427:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727207:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302902.727885:0:18427:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].17, 64 off9352 00000004:00080000:1.0:1789302902.727895:0:18427:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 739738391865ns, next at 740738389469ns, rc = 0 00010000:00010000:1.0:1789302902.727965:0:18427:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727978:0:18427:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727987:0:18427:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.727994:0:18427:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728000:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728006:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728026:0:18427:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302902.728048:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728055:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728061:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728066:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302902.728071:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728078:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728087:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:1.0:1789302902.728090:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302902.728094:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 2/0,0 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728101:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe4309518508db lrc: 0/0,0 mode: --/EX res: [0x200000402:0x30:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728120:0:18427:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728126:0:18427:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728132:0:18427:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728137:0:18427:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302902.728139:0:18427:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728145:0:18427:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728153:0:18427:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302902.728155:0:18427:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302902.728159:0:18427:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728164:0:18427:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe4309518508cd lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302902.728174:0:18427:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000003ea9bd30 reply state 00000000571cb778 transno 4294968390 ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.728184:0:18427:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302902.728254:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70c700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+118:33567:x1876219350595968:12345-192.168.203.11@tcp:36:rm.0 Request processed in 17749us (18933us total) trans 4294968390 rc 0/0 00000100:00100000:1.0:1789302902.728262:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1678 00010000:00010000:1.0:1789302902.728296:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302902.728302:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728309:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302902.728312:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728319:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728327:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x30:0x0].0x0 (ffff960f124a9600) 00010000:00010000:1.0:1789302902.728329:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1789302902.728332:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728337:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe4309518508e2 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x30:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.728369:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302902.728370:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302902.728372:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1789302902.728554:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789302902.746909:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.746931:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.746939:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.746945:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302902.746951:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.746960:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.746979:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x353b91a9 (ffff960f095b2200) 00010000:00010000:0.0:1789302902.746983:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302902.746988:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.746995:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe4309518508d4 lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x353b91a9 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 18427 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302902.948376:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968389 is committed 00000004:00080000:2.0:1789302902.948389:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 21 committed 00000001:00080000:2.0:1789302902.948415:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968385 is committed 00000001:00080000:2.0:1789302902.948458:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968390 is committed 00000004:00080000:2.0:1789302902.948462:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 20 committed 00000100:00100000:3.0:1789302902.948584:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371190784:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789302902.948654:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302902.948733:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371190784 00000100:00100000:3.0:1789302902.948740:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371190784 00000001:00080000:2.0:1789302902.948747:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968386 is committed 00000004:00080000:2.0:1789302902.948799:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 19 committed 00000100:00100000:3.0:1789302902.948807:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000001:00080000:2.0:1789302902.948809:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968387 is committed 00000100:00100000:3.0:1789302902.948820:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371190784:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1789302902.948847:0:6696:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:31 count 1 00010000:00010000:3.0:1789302902.948874:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.948895:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 23 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.948905:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.948913:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1789302902.948918:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302902.948925:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.948955:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 20 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302902.948963:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371190912:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1789302902.948971:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 20 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302902.948984:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371191168:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302902.949005:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:3.0:1789302902.949067:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 23 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.949079:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 23 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302902.949089:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371191296:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302902.953560:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302902.953583:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371191040:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789302902.953669:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302902.953729:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371190912 00000100:00100000:2.0:1789302902.953740:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371190912 00000100:00100000:2.0:1789302902.953793:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371191040 00000100:00100000:2.0:1789302902.953797:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371191040 00000100:00100000:2.0:1789302902.953818:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1789302902.953828:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371190912:12345-0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789302902.953846:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe0c00 going to sleep for 11 seconds 00000100:00080000:3.0:1789302902.953865:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00002000:00080000:2.0:1789302902.953866:0:6695:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:30 count 1 00000100:00080000:3.0:1789302902.953871:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302902.953888:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:3.0:1789302902.953899:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371191040:12345-0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1789302902.953899:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00002000:00080000:3.0:1789302902.953918:0:6697:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:31 count 1 00010000:00010000:2.0:1789302902.953923:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.953934:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 21 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.953937:0:6697:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.953947:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=2s v=(0 0) 00010000:00010000:2.0:1789302902.953955:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302902.953962:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.953963:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.953986:0:6697:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 21 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.953994:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=2s v=(0 0) 00010000:00010000:3.0:1789302902.953997:0:6697:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302902.954001:0:6697:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.954011:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 21 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.954029:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 21 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.954039:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 21 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302902.954044:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371191424:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302902.954058:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 21 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302902.954072:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371191552:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302902.954094:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 17 pid: 6696 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.954105:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 17 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302902.954116:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371191680:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302902.954121:0:6697:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 17 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.954134:0:6697:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 17 pid: 13439 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302902.954142:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1e599c00 going to sleep for 11 seconds 00000100:00100000:3.0:1789302902.954146:0:6697:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371191808:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302902.954174:0:6697:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3efe00c0 going to sleep for 11 seconds 00000100:00100000:1.0:1789302902.969197:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302902.969230:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371191552:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302902.969251:0:6697:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef41880 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0001_UUID:6697:1876219371191808:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302902.969254:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.969276:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.969294:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.969316:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.969352:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302902.969391:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596480 00000100:00100000:1.0:1789302902.969399:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596480 00010000:00010000:1.0:1789302902.969444:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 21 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.969470:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596736 00000100:00100000:1.0:1789302902.969472:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596736 00010000:00010000:1.0:1789302902.969497:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 22 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.969530:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596992 00000100:00100000:1.0:1789302902.969533:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596992 00010000:00010000:1.0:1789302902.969547:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 18 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.969559:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596608 00000100:00100000:1.0:1789302902.969561:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596608 00010000:00010000:1.0:1789302902.969573:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 24 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.969590:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350596864 00000100:00100000:1.0:1789302902.969593:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350596864 00010000:00010000:1.0:1789302902.969602:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 23 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302902.970087:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350597120 00000100:00100000:2.0:1789302902.970093:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350597120 00010000:00010000:2.0:1789302902.970117:0:13636:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 19 pid: 13439 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302902.970147:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00100000:2.0:1789302902.970157:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2348:x1876219350596480:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302902.970172:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302902.970178:0:13636:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 21 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302902.970190:0:13636:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 21 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.970719:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:1.0:1789302902.970730:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+23:2348:x1876219350596736:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302902.970740:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302902.970747:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 23 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.970757:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 23 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.970772:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302902.970830:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+23:2348:x1876219350596736:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 102us (1579us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302902.970838:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 488 00000100:00100000:1.0:1789302902.970858:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:1.0:1789302902.970866:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350596992:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302902.970874:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302902.970879:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 19 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.970895:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 19 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.970908:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302902.970912:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1e:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302902.970918:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.970928:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.970947:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302902.970968:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+19:2348:x1876219350596992:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 103us (1691us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302902.970974:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 489 00000100:00100000:1.0:1789302902.970987:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 490 00000100:00100000:1.0:1789302902.970994:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350596608:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302902.971002:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302902.971007:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 6 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 24 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302902.971665:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00100000:0.0:1789302902.971677:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+22:2347:x1876219350596864:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302902.971686:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302902.971693:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 22 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971705:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 22 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971720:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2ad758c/0x64fe43095185085d lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15cc expref: 22 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971732:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302902.971758:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2347:x1876219350596864:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 84us (2446us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302902.971765:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 491 00000100:00100000:0.0:1789302902.971784:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 492 00000100:00100000:0.0:1789302902.971792:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd05050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+18:2347:x1876219350597120:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302902.971800:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302902.971806:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 18 pid: 13439 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971815:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 18 pid: 13439 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971827:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001b42e3e4/0x64fe43095185014f lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a125a expref: 18 pid: 13439 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971838:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302902.971841:0:5830:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302902.971846:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971854:0:5830:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.971869:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302902.971888:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd05050 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+17:2347:x1876219350597120:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 98us (2535us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302902.971894:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 492 00010000:00010000:3.0:1789302902.972686:0:6697:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.972697:0:6697:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.972709:0:6697:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.972720:0:6697:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.972726:0:6697:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.972746:0:6697:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1789302902.972806:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302902.972854:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2348:x1876219350596480:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 2696us (3672us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302902.972863:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 487 00000100:00080000:2.0:1789302902.972962:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302902.972967:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1789302902.973497:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 24 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.973542:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302902.973546:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x1f:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302902.973551:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.973559:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.973577:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302902.975273:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+7:3300:x1876219371191040:12345-0@lo:6:osp-syn-1-0.0 Request processed in 21369us (21596us total) trans 4294967387 rc 0/0 00000100:00100000:3.0:1789302902.975287:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1789302902.975313:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+24:2347:x1876219350596608:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 2619us (4318us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302902.975328:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 490 00010000:00010000:3.0:1789302902.975336:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302902.975357:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.975370:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302902.975375:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.975388:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.975409:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302902.975418:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000ff4882b5/0x64fe4309518508f7 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6697 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.976164:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000016bcf938/0x64fe43095185083a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15b7 expref: 20 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.976183:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371191424:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302902.976201:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d5238f11/0x64fe430951850109 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x1e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1206 expref: 16 pid: 6696 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.976213:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7b800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371191680:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302902.976665:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302902.976669:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302902.976672:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302902.977666:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.977684:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.977694:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.977717:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.977725:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.977752:0:6695:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1789302902.978319:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3300:x1876219371190912:12345-0@lo:6:osp-syn-1-0.0 Request processed in 24488us (29311us total) trans 4294967388 rc 0/0 00000100:00100000:1.0:1789302902.978329:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000004:00080000:1.0:1789302902.978409:0:3300:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ffbc700/2, rc 0, transno 4294967387 00000100:00100000:1.0:1789302902.978416:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ffbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371191040:0@lo:6:osp-syn-1-0.0 00000004:00080000:1.0:1789302902.978426:0:3300:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e0290b100/2, rc 0, transno 4294967388 00000100:00100000:1.0:1789302902.978429:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371190912:0@lo:6:osp-syn-1-0.0 00010000:00010000:1.0:1789302902.978460:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000766cfb13/0x64fe430951850841 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15be expref: 20 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.978474:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371191168:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302902.978487:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000004707067e/0x64fe430951850148 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x1f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1253 expref: 23 pid: 6697 timeout: 839 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302902.978525:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:6696:1876219371191296:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302902.978542:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.978552:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.978559:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.978568:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.978576:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302902.978592:0:6696:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1789302902.978770:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1789302902.978780:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978790:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302902.978795:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978806:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978826:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978835:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000797bbe74/0x64fe4309518508f0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x1e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978866:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302902.978870:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978877:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302902.978881:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978889:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978902:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978908:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bb0ec6bc/0x64fe4309518508e9 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x1f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302902.978928:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302902.978931:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302902.978933:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302902.979045:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371190784:12345-0@lo:6:osp-syn-0-0.0 Request processed in 30227us (30389us total) trans 4294967390 rc 0/0 00000100:00100000:1.0:1789302902.979054:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00010000:00010000:1.0:1789302902.979075:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789302902.980838:0:3303:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ef50e00/2, rc 0, transno 4294967390 00000100:00100000:3.0:1789302902.980853:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371190784:0@lo:6:osp-syn-0-0.0 00000001:00080000:0.0:1789302902.982173:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967385 is committed 00000001:00080000:0.0:1789302902.982193:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967386 is committed 00000001:00080000:0.0:1789302902.982203:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967387 is committed 00000001:00080000:0.0:1789302902.982231:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967389 is committed 00010000:00010000:3.0:1789302903.444371:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302903.827277:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef41880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371191936:0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.827357:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302903.827414:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371192448:0@lo:400:kworker.0 00000100:00100000:1.0:1789302903.827555:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371192192:0@lo:400:kworker.0 00000100:00100000:1.0:1789302903.827587:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302903.828218:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302903.828255:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1789302903.828263:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371192064:0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.828286:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302903.828306:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371192576:0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.828342:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302903.828389:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371192320:0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.828411:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302903.828532:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371192192 00000100:00100000:2.0:1789302903.828541:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371192192 00000100:00100000:2.0:1789302903.828592:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371192448 00000100:00100000:2.0:1789302903.828595:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371192448 00000100:00100000:2.0:1789302903.828627:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371192064 00000100:00100000:2.0:1789302903.828630:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371192064 00000100:00100000:2.0:1789302903.828644:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 521 00000100:00100000:2.0:1789302903.828653:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371192192:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302903.828705:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c46a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371192192:12345-0@lo:400:kworker.0 Request processed in 52us (1115us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302903.828716:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 521 00000100:00100000:2.0:1789302903.828732:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:2.0:1789302903.828738:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371192448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302903.828773:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371192448:12345-0@lo:400:kworker.0 Request processed in 35us (556us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302903.828778:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000100:00100000:2.0:1789302903.828789:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 523 00000100:00100000:2.0:1789302903.828795:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371192064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302903.828823:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371192064:12345-0@lo:400:kworker.0 Request processed in 28us (535us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302903.828828:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 523 00000100:00080000:2.0:1789302903.828844:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302903.829144:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371191936 00000100:00100000:3.0:1789302903.829149:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371191936 00000100:00100000:3.0:1789302903.829186:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 627 00000100:00100000:3.0:1789302903.829193:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371191936:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.829226:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371191936:12345-0@lo:400:kworker.0 Request processed in 33us (1868us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302903.829232:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 627 00000100:00100000:3.0:1789302903.829262:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371192576 00000100:00100000:3.0:1789302903.829266:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371192576 00000100:00100000:3.0:1789302903.829289:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371192320 00000100:00100000:3.0:1789302903.829292:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371192320 00000100:00100000:3.0:1789302903.829308:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:3.0:1789302903.829313:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371192576:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.829717:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371192576:12345-0@lo:400:kworker.0 Request processed in 402us (1383us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302903.829724:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:3.0:1789302903.829738:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 547 00000100:00100000:3.0:1789302903.829745:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371192320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.829790:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371192320:12345-0@lo:400:kworker.0 Request processed in 46us (1377us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302903.829796:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 547 00000100:00100000:3.0:1789302903.829846:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371192064:0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.829859:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371192576:0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.829868:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef41880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371191936:0@lo:400:kworker.0 00000100:00100000:3.0:1789302903.829877:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371192448:0@lo:400:kworker.0 00000100:00100000:2.0:1789302903.830345:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371192320:0@lo:400:kworker.0 00000100:00100000:1.0:1789302903.832283:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371192192:0@lo:400:kworker.0 00000001:00080000:0.0:1789302903.911153:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967388 is committed 00000001:00080000:0.0:1789302903.911170:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967386 is committed 00000040:00080000:2.0:1789302903.975070:0:22897:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302903.975079:0:22897:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302903.975157:0:22897:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:2.0:1789302903.975162:0:22897:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302904.016834:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302904.019791:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350597248 00000100:00100000:0.0:1789302904.019819:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350597248 00000100:00100000:0.0:1789302904.019943:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 90 00000100:00100000:0.0:1789302904.019963:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2346:x1876219350597248:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302904.020042:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302904.020077:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508fe lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1604 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.020166:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.020188:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.020208:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.020230:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.020244:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.020252:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302904.020449:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302904.020918:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350597504 00000100:00100000:1.0:1789302904.020929:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350597504 00000100:00100000:1.0:1789302904.020978:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1679 00000100:00100000:1.0:1789302904.020996:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+116:33580:x1876219350597504:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:0.0:1789302904.021012:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021022:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021029:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021035:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f121ddc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302904.021040:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021047:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302904.021048:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021064:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302904.021066:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcf1e00/0x64fe43095184ffe3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1196 expref: 116 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021067:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302904.021074:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021080:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850905 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021082:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:1.0:1789302904.021086:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302904.021091:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021098:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021100:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021105:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021109:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302904.021110:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021112:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021119:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021119:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021127:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x996ea506 (ffff960f0cf83900) 00010000:00010000:0.0:1789302904.021130:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302904.021133:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021134:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021140:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185090c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x996ea506 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021141:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021148:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021171:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508fe lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1604 expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021179:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302904.021181:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302904.021184:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe4309518508fe lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1604 expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302904.021198:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f121dca00, rc -2) 00000100:00100000:0.0:1789302904.021284:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2346:x1876219350597248:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 1328us (4465us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302904.021293:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 90 00010000:00010000:1.0:1789302904.021374:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021387:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 57 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021394:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021433:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 57 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302904.021446:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 57 pid: 5844 timeout: 841 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302904.021459:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c47480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371192704:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302904.021571:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f098a2480 going to sleep for 11 seconds 00000100:00080000:2.0:1789302904.022578:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302904.025383:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c47480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371192704:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302904.025436:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302904.028242:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302904.028787:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350597632 00000100:00100000:2.0:1789302904.028803:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350597632 00000100:00080000:0.0:1789302904.028878:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302904.028921:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 58 pid: 5844 timeout: 841 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302904.028976:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 493 00000100:00100000:2.0:1789302904.028992:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+58:2347:x1876219350597632:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302904.029013:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302904.029021:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 58 pid: 5844 timeout: 841 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302904.029039:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 58 pid: 5844 timeout: 841 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302904.029179:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe43095184ff88 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x31:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1111 expref: 58 pid: 5844 timeout: 841 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302904.029200:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:2.0:1789302904.029206:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302904.029217:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302904.029225:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302904.029698:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1789302904.029746:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.029766:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302904.029782:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:2347:x1876219350597632:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 792us (1550us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302904.029794:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 493 00000040:00080000:3.0:1789302904.031064:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].20, 64 off9544 00000004:00080000:3.0:1789302904.031077:0:5845:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 741041574358ns, next at 742041571411ns, rc = 0 00010000:00010000:3.0:1789302904.031162:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031178:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031189:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031197:0:5845:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031203:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031209:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031234:0:5845:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302904.031253:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031259:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031265:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031271:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302904.031275:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031282:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031293:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:3.0:1789302904.031296:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302904.031301:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031306:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe430951850921 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x31:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031325:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031344:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031351:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031357:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbec00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302904.031360:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031366:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031378:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302904.031380:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302904.031383:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031389:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbec00/0x64fe430951850913 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302904.031399:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ac89c8cf reply state 0000000023b70799 transno 4294968391 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302904.031408:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302904.031496:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33580:x1876219350597504:12345-192.168.203.11@tcp:36:rm.0 Request processed in 10498us (11046us total) trans 4294968391 rc 0/0 00000100:00100000:3.0:1789302904.031591:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1679 00010000:00010000:3.0:1789302904.031630:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302904.031636:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031642:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302904.031645:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031652:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031660:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x31:0x0].0x0 (ffff960f0cf83500) 00010000:00010000:3.0:1789302904.031662:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302904.031730:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031741:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2800/0x64fe430951850928 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x31:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.031760:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302904.031761:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302904.031763:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302904.037558:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.037577:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.037585:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.037592:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3fcbe400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302904.037598:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.037607:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.037623:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x54ccc52d (ffff960f2c94d500) 00010000:00010000:3.0:1789302904.037627:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302904.037631:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302904.037638:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fcbe400/0x64fe43095185091a lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x54ccc52d bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302904.333260:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371192832:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302904.333369:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302904.333947:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371192832 00000100:00100000:3.0:1789302904.333959:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371192832 00000100:00100000:3.0:1789302904.334108:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789302904.334122:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371192832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302904.334170:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302904.334240:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c47480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371192832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 121us (872us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302904.334248:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:3.0:1789302904.334300:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967388 gen 2 00000004:00080000:3.0:1789302904.334307:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ffbc700, transno 4294967387 00000004:00080000:3.0:1789302904.334319:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e0290b100, transno 4294967388 00000100:00100000:3.0:1789302904.334346:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25c44a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371192832:0@lo:13:osp-pre-1-0.0 00000100:00080000:1.0:1789302904.335753:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:1.0:1789302904.460941:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302904.846962:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302904.847006:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371192960:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302904.847111:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302904.847429:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371192960 00000100:00100000:0.0:1789302904.847441:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371192960 00000100:00100000:0.0:1789302904.847541:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 225 00000100:00100000:0.0:1789302904.847557:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371192960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302904.847618:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302904.847685:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371192960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 133us (574us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302904.847692:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 225 00000100:00080000:2.0:1789302904.847867:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302904.848728:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371192960:0@lo:13:osp-pre-0-0.0 00000040:00080000:3.0:1789302905.107539:0:22949:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302905.107548:0:22949:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302905.107622:0:22949:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:3.0:1789302905.107629:0:22949:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302905.108008:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.108046:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.108069:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.108100:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.108123:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.108151:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.108184:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.108296:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350597888 00000100:00100000:2.0:1789302905.108305:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350597888 00000100:00100000:2.0:1789302905.108366:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350598272 00000100:00100000:2.0:1789302905.108369:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350598272 00000100:00100000:2.0:1789302905.108398:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 524 00000100:00100000:2.0:1789302905.108410:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2348:x1876219350597888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302905.108582:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2348:x1876219350597888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 81us (484us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.108593:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 524 00000100:00100000:2.0:1789302905.108611:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 525 00000100:00100000:2.0:1789302905.108618:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:2348:x1876219350598272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302905.108649:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:2348:x1876219350598272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.108655:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 525 00000100:00100000:2.0:1789302905.108673:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350598144 00000100:00100000:2.0:1789302905.108677:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350598144 00000100:00100000:2.0:1789302905.108716:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350598400 00000100:00100000:2.0:1789302905.108719:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350598400 00000100:00100000:2.0:1789302905.108736:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350598016 00000100:00100000:2.0:1789302905.108738:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350598016 00000100:00100000:2.0:1789302905.108751:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350598528 00000100:00100000:2.0:1789302905.108754:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350598528 00000100:00100000:2.0:1789302905.108767:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 548 00000100:00100000:2.0:1789302905.108773:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2348:x1876219350598144:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302905.108798:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2348:x1876219350598144:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (754us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.108805:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 548 00000100:00100000:2.0:1789302905.108817:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 549 00000100:00100000:2.0:1789302905.108824:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350598400:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302905.108848:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2348:x1876219350598400:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (778us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.108854:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 549 00000100:00100000:2.0:1789302905.108866:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 550 00000100:00100000:2.0:1789302905.108872:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2348:x1876219350598016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302905.108895:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2348:x1876219350598016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (773us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.108903:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 550 00000100:00100000:2.0:1789302905.108917:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 551 00000100:00100000:2.0:1789302905.108923:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350598528:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302905.108947:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350598528:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (764us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.108953:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 551 00000100:00100000:2.0:1789302905.108977:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350597760 00000100:00100000:2.0:1789302905.108981:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350597760 00000100:00100000:2.0:1789302905.109009:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 628 00000100:00100000:2.0:1789302905.109016:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350597760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302905.109176:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350597760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 159us (1073us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.109183:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 628 00000100:00080000:0.0:1789302905.111753:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302905.111762:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302905.226002:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.226082:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350598656 00000100:00100000:2.0:1789302905.226092:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350598656 00000100:00100000:2.0:1789302905.226155:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 91 00000100:00100000:2.0:1789302905.226166:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2345:x1876219350598656:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:2.0:1789302905.226203:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302905.226224:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185092f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a160b expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226283:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226297:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226309:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226335:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226342:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226349:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226418:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226426:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226433:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226440:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302905.226444:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226453:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226464:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302905.226467:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302905.226472:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226479:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850936 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226490:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226496:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226530:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226535:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302905.226538:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226544:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226551:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xbb834312 (ffff960f095b2d00) 00010000:00010000:2.0:1789302905.226554:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302905.226556:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226563:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185093d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226593:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185092f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a160b expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226602:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302905.226605:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789302905.226610:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe43095185092f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a160b expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.226619:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc -2) 00000100:00100000:2.0:1789302905.226691:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2345:x1876219350598656:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 526us (698us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302905.226699:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 91 00000100:00100000:3.0:1789302905.231644:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302905.231721:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350598912 00000100:00100000:0.0:1789302905.231731:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350598912 00000100:00100000:0.0:1789302905.231791:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1680 00000100:00100000:0.0:1789302905.231802:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33590:x1876219350598912:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:0.0:1789302905.231854:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302905.231880:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850125 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a120d expref: 114 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.231900:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:0.0:1789302905.231905:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302905.231928:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.231942:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.231951:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.231965:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.231972:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.232072:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.232171:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.232184:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 56 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.232191:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.232223:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 56 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302905.232237:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 56 pid: 5845 timeout: 842 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302905.232251:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371193088:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302905.232382:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fdba80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302905.242337:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371193088:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302905.242367:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302905.242730:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302905.242806:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350599040 00000100:00100000:2.0:1789302905.242813:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350599040 00010000:00010000:2.0:1789302905.242851:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 57 pid: 5845 timeout: 842 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302905.242885:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 494 00000100:00100000:2.0:1789302905.242895:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+57:2347:x1876219350599040:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302905.242909:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302905.242916:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 57 pid: 5845 timeout: 842 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.242928:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 57 pid: 5845 timeout: 842 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.242949:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe430951850099 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x32:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a11dc expref: 57 pid: 5845 timeout: 842 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.242964:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:2.0:1789302905.242969:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.242988:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.242996:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.243023:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302905.243072:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2347:x1876219350599040:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 177us (341us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302905.243083:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 494 00010000:00010000:3.0:1789302905.243252:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.243260:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302905.243948:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].18, 64 off9416 00000004:00080000:3.0:1789302905.243957:0:5845:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 742254454493ns, next at 743254452301ns, rc = 0 00010000:00010000:3.0:1789302905.244255:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244270:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244280:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244287:0:5845:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244294:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244300:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244333:0:5845:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302905.244353:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244360:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244366:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244372:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302905.244376:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244383:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244392:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:3.0:1789302905.244394:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302905.244398:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244403:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850952 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x32:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244418:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244423:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244428:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244432:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302905.244435:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244440:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244450:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302905.244451:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302905.244454:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.244458:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850944 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302905.244466:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000009c0fe5d1 reply state 000000003c85142a transno 4294968392 ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302905.244474:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302905.244569:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33590:x1876219350598912:12345-192.168.203.11@tcp:36:rm.0 Request processed in 12767us (12931us total) trans 4294968392 rc 0/0 00000100:00100000:3.0:1789302905.244577:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1680 00010000:00010000:2.0:1789302905.245523:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1789302905.245531:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.245538:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:2.0:1789302905.245542:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.245550:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.245561:0:5833:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x32:0x0].0x0 (ffff960f124a9b00) 00010000:00010000:2.0:1789302905.245564:0:5833:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1789302905.245568:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.245573:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850959 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x32:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302905.245590:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302905.245592:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302905.245593:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302905.247993:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302905.248011:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302905.248019:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302905.248025:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302905.248030:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302905.248037:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302905.248052:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xde5271db (ffff960f0cf83900) 00010000:00010000:1.0:1789302905.248055:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302905.248060:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302905.248066:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185094b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0xde5271db bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302905.484697:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302906.354582:0:23000:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302906.354594:0:23000:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302906.354676:0:23000:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:3.0:1789302906.354684:0:23000:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302906.405172:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302906.405396:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350599168 00000100:00100000:2.0:1789302906.405409:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350599168 00000100:00100000:2.0:1789302906.405487:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 92 00000100:00100000:2.0:1789302906.405519:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2346:x1876219350599168:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:2.0:1789302906.405572:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302906.405599:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850960 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1620 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.407489:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.407536:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.407556:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.407575:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.407583:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.407591:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302906.409263:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00010000:00010000:2.0:1789302906.409367:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409382:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409391:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409399:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302906.409404:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409418:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409438:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302906.409441:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302906.409448:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409455:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517200/0x64fe430951850967 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409471:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409478:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409485:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409490:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302906.409493:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409499:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409527:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x4b66ad81 (ffff960f095b2d00) 00010000:00010000:2.0:1789302906.409530:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302906.409533:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409545:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516200/0x64fe43095185096e lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x4b66ad81 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409597:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850960 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1620 expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409609:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302906.409611:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789302906.409625:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850960 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1620 expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.409637:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01516000, rc -2) 00000100:00100000:2.0:1789302906.409740:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2346:x1876219350599168:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 4242us (4580us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302906.409751:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 92 00000100:00080000:2.0:1789302906.409793:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302906.409815:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350599424 00000100:00100000:2.0:1789302906.409825:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350599424 00000100:00100000:2.0:1789302906.409871:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1681 00000100:00100000:2.0:1789302906.409881:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33602:x1876219350599424:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:2.0:1789302906.412333:0:5844:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302906.412381:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6800/0x64fe430951850308 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12f4 expref: 114 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.412405:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:2.0:1789302906.412412:0:5844:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1789302906.412444:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.412462:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.412473:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.412493:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.413748:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.413769:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.413901:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.413918:0:5844:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 55 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.413928:0:5844:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.414831:0:5844:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 55 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302906.414854:0:5844:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 55 pid: 18427 timeout: 843 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302906.414878:0:5844:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371193216:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302906.415868:0:5844:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0311da80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302906.428480:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302906.428595:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350599552 00000100:00100000:0.0:1789302906.428608:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350599552 00010000:00010000:0.0:1789302906.428722:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 56 pid: 18427 timeout: 843 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302906.428766:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 495 00000100:00100000:0.0:1789302906.428778:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2347:x1876219350599552:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302906.428800:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302906.428809:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 56 pid: 18427 timeout: 843 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302906.428824:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 56 pid: 18427 timeout: 843 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302906.428855:0:5831:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:0.0:1789302906.428861:0:5831:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302906.428873:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302906.428882:0:5831:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302906.428906:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302906.428974:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+56:2347:x1876219350599552:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 198us (510us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302906.428982:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 495 00010000:00010000:3.0:1789302906.429556:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8800/0x64fe43095185029f lrc: 0/0,0 mode: --/PR res: [0x200000402:0x33:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12c3 expref: 55 pid: 18427 timeout: 843 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302906.429579:0:5844:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000_UUID:5844:1876219371193216:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302906.429606:0:5844:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.429617:0:5844:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.429626:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:3.0:1789302906.430579:0:5844:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].21, 64 off9608 00000004:00080000:3.0:1789302906.430593:0:5844:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 743441089831ns, next at 744441085956ns, rc = 0 00010000:00010000:3.0:1789302906.430679:0:5844:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 3/0,1 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430695:0:5844:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430707:0:5844:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430717:0:5844:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430724:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430729:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430750:0:5844:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:3.0:1789302906.430776:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430783:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430791:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.430799:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302906.431032:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431043:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431057:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:3.0:1789302906.431060:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302906.431065:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431072:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6a00/0x64fe430951850983 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x33:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431090:0:5844:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431097:0:5844:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431104:0:5844:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431108:0:5844:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302906.431111:0:5844:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431117:0:5844:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431129:0:5844:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302906.431131:0:5844:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302906.431134:0:5844:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.431140:0:5844:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850975 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:3.0:1789302906.431151:0:5844:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000ce14c230 reply state 00000000174d9b21 transno 4294968393 ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302906.431160:0:5844:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:3.0:1789302906.431246:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33602:x1876219350599424:12345-192.168.203.11@tcp:36:rm.0 Request processed in 21363us (21994us total) trans 4294968393 rc 0/0 00000100:00100000:3.0:1789302906.431256:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1681 00010000:00010000:3.0:1789302906.431967:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302906.432036:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.432044:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf3c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302906.432048:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.432057:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.432070:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x33:0x0].0x0 (ffff960f2c94d800) 00010000:00010000:3.0:1789302906.432073:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302906.432077:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 1/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.432088:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe43095185098a lrc: 0/0,0 mode: --/PW res: [0x200000402:0x33:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.432107:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302906.432109:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302906.432111:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1789302906.432264:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302906.452823:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.452846:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.452855:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.452861:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f120dda00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302906.452867:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.452875:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.452895:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x10f4e958 (ffff960f095b2100) 00010000:00010000:3.0:1789302906.452899:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302906.452904:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.452911:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dda00/0x64fe43095185097c lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x10f4e958 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5844 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302906.508913:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302907.461681:0:23052:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302907.461693:0:23052:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302907.461792:0:23052:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:0.0:1789302907.461805:0:23052:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302907.533649:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302907.559235:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302907.559374:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350599680 00000100:00100000:0.0:1789302907.559480:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350599680 00000100:00100000:0.0:1789302907.559556:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 93 00000100:00100000:0.0:1789302907.559569:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2348:x1876219350599680:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302907.559618:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302907.559645:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850991 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1627 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559718:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559736:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559749:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559767:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559775:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559782:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559855:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559863:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559871:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559879:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302907.559885:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559893:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559908:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302907.559911:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302907.559917:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559925:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850998 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559938:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559945:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.559951:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.561292:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302907.561298:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.561316:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.561332:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xbb834312 (ffff960f0cf83200) 00010000:00010000:0.0:1789302907.561335:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302907.561340:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.561347:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe43095185099f lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.561607:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850991 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1627 expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.561620:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302907.561623:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302907.561627:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850991 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1627 expref: 115 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.561637:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f25f72a00, rc -2) 00000100:00100000:1.0:1789302907.566467:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:2348:x1876219350599680:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 5807us (6152us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302907.566487:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 93 00000100:00100000:0.0:1789302907.573373:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302907.573527:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350599936 00000100:00100000:1.0:1789302907.573539:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350599936 00000100:00100000:1.0:1789302907.573603:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1682 00000100:00100000:1.0:1789302907.573614:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+114:33612:x1876219350599936:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:1.0:1789302907.573686:0:6919:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302907.573720:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518503e8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1348 expref: 114 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573750:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:1.0:1789302907.573756:0:6919:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302907.573786:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573803:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573815:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573832:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573839:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573846:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573962:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573972:0:6919:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 54 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.573982:0:6919:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.574024:0:6919:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 54 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.574042:0:6919:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 54 pid: 5844 timeout: 844 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302907.574059:0:6919:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371193344:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302907.574315:0:6919:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f03f94d80 going to sleep for 11 seconds 00000100:00100000:1.0:1789302907.591171:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302907.591345:0:6919:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41fbb800 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:6919:1876219371193344:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302907.591407:0:6919:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302907.591786:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600064 00000100:00100000:0.0:1789302907.591797:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600064 00010000:00010000:0.0:1789302907.591847:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 55 pid: 5844 timeout: 844 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302907.591900:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 496 00000100:00100000:0.0:1789302907.591911:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+55:2347:x1876219350600064:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302907.591929:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302907.591938:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 55 pid: 5844 timeout: 844 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.591968:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 55 pid: 5844 timeout: 844 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.591995:0:5830:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe4309518503b7 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x34:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1333 expref: 55 pid: 5844 timeout: 844 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.592016:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:0.0:1789302907.592021:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.592032:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.592042:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302907.592075:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302907.592142:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2347:x1876219350600064:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 233us (984us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302907.592485:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 496 00010000:00010000:1.0:1789302907.593928:0:6919:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.593947:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302907.599181:0:6919:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].19, 64 off9480 00000004:00080000:1.0:1789302907.599205:0:6919:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 744609701753ns, next at 745609692273ns, rc = 0 00010000:00010000:1.0:1789302907.599319:0:6919:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 3/0,1 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599343:0:6919:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599358:0:6919:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599369:0:6919:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599376:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599397:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599419:0:6919:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302907.599448:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599454:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599461:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599467:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345bc00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302907.599473:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599481:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599495:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:1.0:1789302907.599498:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302907.599525:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599533:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345bc00/0x64fe4309518509b4 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x34:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599556:0:6919:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599562:0:6919:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599569:0:6919:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599575:0:6919:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302907.599578:0:6919:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599586:0:6919:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599593:0:6919:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302907.599596:0:6919:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302907.599598:0:6919:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302907.599604:0:6919:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509a6 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302907.599615:0:6919:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000008ae811b1 reply state 00000000163e7d60 transno 4294968394 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302907.599626:0:6919:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302907.599738:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+115:33612:x1876219350599936:12345-192.168.203.11@tcp:36:rm.0 Request processed in 26119us (26379us total) trans 4294968394 rc 0/0 00000100:00100000:1.0:1789302907.599750:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1682 00010000:00010000:3.0:1789302907.601683:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302907.601698:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302907.601710:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302907.601716:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302907.601728:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302907.601748:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x34:0x0].0x0 (ffff960f10d3c800) 00010000:00010000:3.0:1789302907.601752:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302907.601757:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 1/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302907.601777:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509bb lrc: 0/0,0 mode: --/PW res: [0x200000402:0x34:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302907.601805:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302907.601808:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302907.601809:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302907.611034:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302907.611063:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302907.611072:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302907.611079:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302907.611085:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302907.611094:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302907.611115:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x2b119043 (ffff960f2c94da00) 00010000:00010000:2.0:1789302907.611118:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302907.611124:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302907.611131:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe4309518509ad lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x2b119043 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 6919 initiator: MDT0 lru_score: 0 lru_type: 0 00000001:00080000:2.0:1789302908.035219:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968394 is committed 00000004:00080000:2.0:1789302908.035233:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 22 committed 00000004:00080000:2.0:1789302908.035256:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 23 committed 00000004:00080000:2.0:1789302908.035269:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 21 committed 00000004:00080000:2.0:1789302908.035281:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 22 committed 00000100:00100000:1.0:1789302908.035753:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371193472:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789302908.035823:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302908.035879:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302908.035899:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371193600:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1789302908.035923:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302908.035997:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371193472 00000100:00100000:2.0:1789302908.036005:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371193472 00000100:00100000:2.0:1789302908.036059:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371193600 00000100:00100000:2.0:1789302908.036062:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371193600 00000100:00100000:2.0:1789302908.036082:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:2.0:1789302908.036090:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371193472:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1789302908.036114:0:6695:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:32 count 1 00010000:00010000:2.0:1789302908.036138:0:6695:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036160:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036170:0:6695:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 19 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036177:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=8s v=(0 0) 00010000:00010000:2.0:1789302908.036181:0:6695:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302908.036186:0:6695:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036212:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 19 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036226:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 19 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302908.036238:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371193728:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302908.036314:0:6695:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 11 pid: 8571 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036326:0:6695:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 11 pid: 8571 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302908.036335:0:6695:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371193856:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302908.036357:0:6695:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ab25e40 going to sleep for 11 seconds 00000100:00100000:2.0:1789302908.036391:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:2.0:1789302908.036399:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+6:3302:x1876219371193600:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:2.0:1789302908.036411:0:6696:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:33 count 1 00010000:00010000:2.0:1789302908.036423:0:6696:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036435:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036444:0:6696:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036471:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=8s v=(0 0) 00010000:00010000:2.0:1789302908.036474:0:6696:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302908.036478:0:6696:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036496:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036538:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302908.036548:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371193984:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302908.036581:0:6696:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 11 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302908.036590:0:6696:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 11 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302908.036599:0:6696:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f11de4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371194112:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302908.036618:0:6696:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f0ab25d80 going to sleep for 11 seconds 00000100:00080000:1.0:1789302908.038101:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302908.039095:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f754e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371194368:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789302908.039154:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302908.039191:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302908.039198:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371194240:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1789302908.039218:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302908.039250:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371194368 00000100:00100000:1.0:1789302908.039257:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371194368 00000100:00100000:1.0:1789302908.039300:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371194240 00000100:00100000:1.0:1789302908.039314:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371194240 00000100:00100000:1.0:1789302908.039337:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:1.0:1789302908.039345:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371194368:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1789302908.041050:0:13439:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:33 count 1 00010000:00010000:1.0:1789302908.041084:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.041106:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.041116:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.041123:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=8s v=(0 0) 00010000:00010000:1.0:1789302908.041127:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302908.041132:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.041173:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.041186:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302908.041199:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371194496:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302908.041293:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 12 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.041314:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 12 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302908.041324:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371194624:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302908.041354:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f1d5b3480 going to sleep for 11 seconds 00000100:00100000:0.0:1789302908.046283:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:0.0:1789302908.046302:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371194240:12345-0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789302908.046813:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302908.046837:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302908.046864:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302908.046893:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302908.046934:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302908.046970:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302908.046988:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600192 00000100:00100000:3.0:1789302908.046997:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600192 00010000:00010000:3.0:1789302908.047039:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 20 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.047066:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600448 00000100:00100000:3.0:1789302908.047069:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600448 00010000:00010000:3.0:1789302908.047082:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 21 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.047093:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600320 00000100:00100000:3.0:1789302908.047095:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600320 00010000:00010000:3.0:1789302908.047109:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 12 pid: 8571 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.047121:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600576 00000100:00100000:3.0:1789302908.047124:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600576 00010000:00010000:3.0:1789302908.047135:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 13 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.047145:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600704 00000100:00100000:3.0:1789302908.047148:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600704 00010000:00010000:3.0:1789302908.047163:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 20 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.047174:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600832 00000100:00100000:3.0:1789302908.047176:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600832 00010000:00010000:3.0:1789302908.047187:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 13 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.047201:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 497 00000100:00100000:3.0:1789302908.047208:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2345:x1876219350600192:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302908.047222:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302908.047227:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 21 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047241:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 21 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047266:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302908.047327:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+21:2345:x1876219350600192:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 119us (518us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.047336:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 497 00000100:00100000:3.0:1789302908.047355:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:3.0:1789302908.047362:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2345:x1876219350600448:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302908.047370:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302908.047374:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 20 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047390:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 20 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047402:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302908.047418:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2345:x1876219350600448:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 57us (581us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.047424:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 498 00000100:00100000:3.0:1789302908.047437:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 499 00000100:00100000:3.0:1789302908.047442:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd04850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2345:x1876219350600320:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302908.047449:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302908.047453:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 13 pid: 8571 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047476:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 13 pid: 8571 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047489:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302908.047492:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302908.047498:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047531:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047550:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302908.047567:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd04850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2345:x1876219350600320:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 125us (703us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.047573:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 499 00000100:00100000:3.0:1789302908.047620:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:3.0:1789302908.047628:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350600576:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302908.047637:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302908.047643:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 12 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047660:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 12 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047673:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302908.047677:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302908.047681:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047689:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047702:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302908.047722:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350600576:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 95us (828us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.047729:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 500 00000100:00100000:3.0:1789302908.047745:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00100000:3.0:1789302908.047752:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2347:x1876219350600704:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302908.047761:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302908.047765:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 20 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047775:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 20 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047795:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302908.047814:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+20:2347:x1876219350600704:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 63us (879us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.047820:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 501 00000100:00100000:3.0:1789302908.047837:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 502 00000100:00100000:3.0:1789302908.047845:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350600832:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302908.047853:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302908.047858:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 5/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 13 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047869:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 13 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047882:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302908.047886:0:5829:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x21:0x0].0x0 since=0s v=(0 0) 00010000:00010000:3.0:1789302908.047890:0:5829:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047898:0:5829:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.047913:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302908.047933:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+13:2347:x1876219350600832:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 88us (961us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.047944:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 502 00000100:00080000:3.0:1789302908.047972:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302908.047975:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789302908.048119:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e41f0adc/0x64fe430951850864 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e1 expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.048132:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad46300 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371193984:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302908.048148:0:6696:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007cf4a66c/0x64fe430951850761 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1571 expref: 11 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.048158:0:6696:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f11de4e00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:6696:1876219371194112:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302908.048187:0:6696:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.048197:0:6696:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.048204:0:6696:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.048214:0:6696:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.048221:0:6696:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.048244:0:6696:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1789302908.050147:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371193600:12345-0@lo:6:osp-syn-1-0.0 Request processed in 13745us (14218us total) trans 4294967389 rc 0/0 00000100:00100000:3.0:1789302908.050158:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00010000:00010000:3.0:1789302908.050196:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302908.050204:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.050215:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef6800 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302908.050220:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.050232:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.050251:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.050259:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000020334217/0x64fe4309518509c9 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6696 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.050278:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302908.050280:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302908.050282:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:3.0:1789302908.050329:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ad44700/2, rc 0, transno 4294967389 00000100:00100000:3.0:1789302908.050337:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371193600:0@lo:6:osp-syn-1-0.0 00010000:00010000:3.0:1789302908.050374:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e588a72d/0x64fe43095185086b lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15d3 expref: 18 pid: 6695 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00002000:00080000:0.0:1789302908.050665:0:18578:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:32 count 1 00010000:00010000:0.0:1789302908.050690:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.050706:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.050716:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.050724:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=8s v=(0 0) 00010000:00010000:0.0:1789302908.050729:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302908.050734:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.050768:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 19 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.050785:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302908.050799:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371194752:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302908.050894:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 12 pid: 13439 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.050911:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 12 pid: 13439 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302908.050921:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371194880:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302908.050949:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f3358aa80 going to sleep for 11 seconds 00000100:00100000:3.0:1789302908.051151:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371193728:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302908.051169:0:6695:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000011fd2e9c/0x64fe4309518502bb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12d8 expref: 10 pid: 8571 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.051177:0:6695:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44000 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0001_UUID:6695:1876219371193856:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302908.051197:0:6695:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.051207:0:6695:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.051214:0:6695:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.051224:0:6695:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.051231:0:6695:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.051253:0:6695:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302908.056950:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b6351f7f/0x64fe43095185084f lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15da expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302908.056986:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9f80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371194496:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302908.057004:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b444b6a2/0x64fe43095185075a lrc: 0/0,0 mode: --/PR res: [0x240000400:0x21:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a156a expref: 12 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302908.057014:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c9c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:13439:1876219371194624:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302908.057043:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057048:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302908.057054:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057058:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.057064:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057077:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7c00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302908.057082:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057085:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.057098:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057102:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.057120:0:13439:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789302908.057124:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057134:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e9bed7d7/0x64fe4309518509c2 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6695 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057153:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302908.057157:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057165:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302908.057168:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057176:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057187:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057194:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000077698a76/0x64fe4309518509d0 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x21:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.057209:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302908.057212:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302908.057215:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302908.057755:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+8:3301:x1876219371194368:12345-0@lo:6:osp-syn-0-0.0 Request processed in 18408us (18598us total) trans 4294967391 rc 0/0 00000100:00100000:1.0:1789302908.057764:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000004:00080000:1.0:1789302908.057816:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f754e00/2, rc 0, transno 4294967391 00000100:00100000:1.0:1789302908.057824:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f754e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371194368:0@lo:6:osp-syn-0-0.0 00000100:00100000:3.0:1789302908.058196:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371193472:12345-0@lo:6:osp-syn-1-0.0 Request processed in 22106us (22371us total) trans 4294967390 rc 0/0 00000100:00100000:3.0:1789302908.058204:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00010000:00010000:3.0:1789302908.058226:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1789302908.058816:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967390 is committed 00000100:00100000:0.0:1789302908.060333:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371194752:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302908.060346:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371194880:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302908.060369:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.060629:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000004:00080000:1.0:1789302908.061397:0:3302:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ad47b80/2, rc 0, transno 4294967390 00000100:00100000:1.0:1789302908.061406:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad47b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371193472:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789302908.062066:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302908.062109:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350601088 00000100:00100000:3.0:1789302908.062117:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350601088 00010000:00010000:3.0:1789302908.062154:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 12 pid: 13439 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.062189:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350600960 00000100:00100000:3.0:1789302908.062192:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350600960 00010000:00010000:3.0:1789302908.062206:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.062226:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 503 00000100:00100000:3.0:1789302908.062233:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+12:2348:x1876219350601088:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302908.062257:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302908.062263:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 12 pid: 13439 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.062274:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 12 pid: 13439 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.062297:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000dfdf86e7/0x64fe4309518502c2 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a12df expref: 12 pid: 13439 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.062330:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302908.063105:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 504 00000100:00100000:0.0:1789302908.063119:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+19:2348:x1876219350600960:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302908.063132:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302908.063138:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063151:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063168:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007623b30d/0x64fe430951850848 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x20:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15c5 expref: 19 pid: 6697 timeout: 845 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063180:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302908.063185:0:8463:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x20:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302908.063190:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063200:0:8463:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063222:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302908.063234:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302908.063249:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350600960:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 133us (1183us total) trans 0 rc 0/0 00010000:00010000:1.0:1789302908.063256:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302908.063260:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 504 00010000:00010000:1.0:1789302908.063274:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.063284:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.063319:0:18578:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1789302908.063580:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302908.063588:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063602:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f121ddc00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302908.063607:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063617:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063631:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063639:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006a8da7e2/0x64fe4309518509d7 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x20:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.063656:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302908.063659:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302908.063660:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302908.063825:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350601088:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 1589us (3195us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.063835:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 503 00000100:00100000:1.0:1789302908.068226:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3301:x1876219371194240:12345-0@lo:6:osp-syn-0-0.0 Request processed in 21921us (28996us total) trans 4294967392 rc 0/0 00000100:00100000:1.0:1789302908.068240:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00100000:0.0:1789302908.068635:0:3301:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967390 gen 2 00000004:00080000:0.0:1789302908.068643:0:3301:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ef50e00, transno 4294967390 00000100:00100000:0.0:1789302908.068666:0:3301:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff960f3f754e00 x1876219371194368/t4294967391(4294967391) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789302924 ref 1 fl Complete:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:0.0:1789302908.068677:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3f755500/2, rc 0, transno 4294967392 00000100:00100000:0.0:1789302908.068683:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371194240:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1789302908.557705:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302908.645127:0:23104:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302908.645135:0:23104:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302908.645219:0:23104:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:2.0:1789302908.645225:0:23104:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302908.787570:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302908.787603:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350601216 00000100:00100000:1.0:1789302908.787612:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350601216 00000100:00100000:1.0:1789302908.787675:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 94 00000100:00100000:1.0:1789302908.787687:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2346:x1876219350601216:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:1.0:1789302908.787725:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302908.787747:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe4309518509de lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a164a expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.787806:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.787821:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.787832:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.787846:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.787852:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.787858:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788051:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788059:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788065:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788071:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302908.788076:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788083:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788094:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302908.788097:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302908.788101:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788107:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe4309518509e5 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788120:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788126:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788132:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788136:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345b400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302908.788139:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788144:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788149:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x71bff0c5 (ffff960f2c94df00) 00010000:00010000:1.0:1789302908.788152:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302908.788154:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788160:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b400/0x64fe4309518509ec lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x71bff0c5 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788186:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe4309518509de lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a164a expref: 111 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788201:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302908.788204:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302908.788208:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ae00/0x64fe4309518509de lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a164a expref: 111 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788217:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345ae00, rc -2) 00000100:00100000:1.0:1789302908.788281:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2346:x1876219350601216:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 596us (719us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302908.788290:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 94 00000100:00080000:1.0:1789302908.788328:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302908.788428:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302908.788532:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350601472 00000100:00100000:1.0:1789302908.788537:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350601472 00000100:00100000:1.0:1789302908.788569:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1683 00000100:00100000:1.0:1789302908.788576:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33625:x1876219350601472:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:1.0:1789302908.788606:0:5845:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302908.788619:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8600/0x64fe4309518504e4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13db expref: 110 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788630:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302908.788634:0:5845:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1789302908.788645:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788654:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788666:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788677:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788684:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788691:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788870:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788880:0:5845:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788888:0:5845:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788918:0:5845:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 53 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.788931:0:5845:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 53 pid: 18427 timeout: 845 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302908.788945:0:5845:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371195008:192.168.203.11@tcp:104: 00000100:00100000:1.0:1789302908.789011:0:5845:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f053b3900 going to sleep for 11 seconds 00000100:00100000:1.0:1789302908.802457:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302908.802598:0:5830:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350601600 00000100:00100000:0.0:1789302908.802607:0:5830:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350601600 00010000:00010000:0.0:1789302908.802662:0:5830:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 54 pid: 18427 timeout: 845 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302908.802694:0:5830:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 505 00000100:00100000:0.0:1789302908.802705:0:5830:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2348:x1876219350601600:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789302908.802719:0:5830:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302908.802726:0:5830:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 5/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 54 pid: 18427 timeout: 845 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.802739:0:5830:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 54 pid: 18427 timeout: 845 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.802760:0:5830:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:0.0:1789302908.802766:0:5830:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.802776:0:5830:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.802783:0:5830:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT flags: 0x50290000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302908.802804:0:5830:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789302908.802849:0:5830:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+54:2348:x1876219350601600:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 146us (409us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302908.802857:0:5830:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 505 00000100:00080000:1.0:1789302908.803049:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302908.808283:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b9000/0x64fe4309518504b3 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x35:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a13c6 expref: 53 pid: 18427 timeout: 845 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302908.808318:0:5845:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000_UUID:5845:1876219371195008:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302908.808346:0:5845:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.808610:0:5845:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.808618:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302908.809578:0:5845:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x3:0x0].22, 64 off9672 00000004:00080000:1.0:1789302908.809591:0:5845:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 745820087658ns, next at 746820084663ns, rc = 0 00010000:00010000:1.0:1789302908.809669:0:5845:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809683:0:5845:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809693:0:5845:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809701:0:5845:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809707:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809713:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809736:0:5845:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302908.809757:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809764:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809771:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809777:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302908.809781:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809789:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809800:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:1.0:1789302908.809803:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302908.809808:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809814:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a600/0x64fe430951850a01 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x35:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809836:0:5845:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809842:0:5845:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809848:0:5845:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809853:0:5845:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a800 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302908.809855:0:5845:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809865:0:5845:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809873:0:5845:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302908.809876:0:5845:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302908.809879:0:5845:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302908.809884:0:5845:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a800/0x64fe4309518509f3 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302908.809895:0:5845:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 00000000b055da85 reply state 000000000a7b9d83 transno 4294968395 ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302908.809902:0:5845:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302908.809987:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33625:x1876219350601472:12345-192.168.203.11@tcp:36:rm.0 Request processed in 21408us (21555us total) trans 4294968395 rc 0/0 00000100:00100000:1.0:1789302908.809996:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1683 00010000:00010000:3.0:1789302908.810595:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1789302908.810604:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.810610:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:3.0:1789302908.810613:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.810619:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.810629:0:17746:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x35:0x0].0x0 (ffff960f2c94d000) 00010000:00010000:3.0:1789302908.810631:0:17746:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1789302908.810634:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.810643:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a08 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x35:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.810659:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302908.810662:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302908.810663:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302908.815398:0:3287:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.815417:0:3287:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.815424:0:3287:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.815431:0:3287:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345be00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302908.815436:0:3287:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.815626:0:3287:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.815643:0:3287:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x6eee4067 (ffff960f2c94d200) 00010000:00010000:3.0:1789302908.815646:0:3287:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302908.815651:0:3287:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302908.815658:0:3287:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe4309518509fa lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x6eee4067 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5845 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302908.962764:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371195136:0@lo:400:kworker.0 00000100:00100000:1.0:1789302908.962829:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302908.962894:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371195136 00000100:00100000:1.0:1789302908.962901:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371195136 00000100:00100000:1.0:1789302908.962943:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:1.0:1789302908.962952:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371195136:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302908.963000:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371195136:12345-0@lo:400:kworker.0 Request processed in 49us (171us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302908.963007:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00100000:1.0:1789302908.963049:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371195136:0@lo:400:kworker.0 00000100:00100000:1.0:1789302908.963081:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302908.963087:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371195264:0@lo:400:kworker.0 00000100:00100000:1.0:1789302908.963112:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302908.963161:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371195392:0@lo:400:kworker.0 00000100:00100000:3.0:1789302908.963671:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371195264 00000100:00100000:3.0:1789302908.963677:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371195264 00000100:00100000:3.0:1789302908.963706:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:3.0:1789302908.963713:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371195264:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302908.963757:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371195264:12345-0@lo:400:kworker.0 Request processed in 43us (643us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302908.963763:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000100:00100000:1.0:1789302908.964283:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302908.965632:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371195520:0@lo:400:kworker.0 00000100:00100000:0.0:1789302908.965680:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302908.965946:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371195264:0@lo:400:kworker.0 00000100:00100000:1.0:1789302908.965974:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371195392 00000100:00100000:1.0:1789302908.965979:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371195392 00000100:00100000:1.0:1789302908.966032:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371195520 00000100:00100000:1.0:1789302908.966035:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371195520 00000100:00100000:1.0:1789302908.966060:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1789302908.966068:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371195392:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302908.966108:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371195392:12345-0@lo:400:kworker.0 Request processed in 42us (2920us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302908.966114:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1789302908.966128:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 528 00000100:00100000:1.0:1789302908.966132:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371195520:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302908.966158:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3301:x1876219371195520:12345-0@lo:400:kworker.0 Request processed in 25us (478us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302908.966163:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 528 00000100:00100000:1.0:1789302908.966186:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371195392:0@lo:400:kworker.0 00000100:00080000:0.0:1789302908.967763:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302908.967941:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3301:1876219371195520:0@lo:400:kworker.0 00000001:00080000:0.0:1789302909.021720:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967390 is committed 00000100:00100000:3.0:1789302909.456474:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371195648:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302909.456575:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302909.456729:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371195648 00000100:00100000:1.0:1789302909.456741:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371195648 00000100:00100000:1.0:1789302909.456798:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:1.0:1789302909.456809:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371195648:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302909.456852:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302909.456901:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371195648:12345-0@lo:13:osp-pre-1-0.0 Request processed in 94us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302909.456908:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:3.0:1789302909.456980:0:3302:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967390 gen 2 00000004:00080000:3.0:1789302909.456986:0:3302:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ad44700, transno 4294967389 00000004:00080000:3.0:1789302909.457006:0:3302:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ad47b80, transno 4294967390 00000100:00100000:3.0:1789302909.457019:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371195648:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302909.581677:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302909.870129:0:23155:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302909.870138:0:23155:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302909.870215:0:23155:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 55 count 11 00000040:00080000:2.0:1789302909.870222:0:23155:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000004:00080000:1.0:1789302909.964814:0:6999:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 746975308817ns, next at 747975306892ns, rc = 0 00000100:00100000:1.0:1789302909.964836:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302909.964854:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371195776:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302909.964928:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302909.964988:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371195776 00000100:00100000:1.0:1789302909.964995:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371195776 00000100:00100000:1.0:1789302909.965040:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1789302909.965049:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371195776:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302909.965082:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302909.965125:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b0b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+7:3302:x1876219371195776:12345-0@lo:13:osp-pre-0-0.0 Request processed in 77us (198us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302909.965132:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 227 00000100:00100000:1.0:1789302909.965177:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371195776:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302910.021258:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302910.021452:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350601856 00000100:00100000:0.0:1789302910.021462:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350601856 00000100:00080000:1.0:1789302910.021529:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302910.021542:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 95 00000100:00100000:0.0:1789302910.021553:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2345:x1876219350601856:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:0.0:1789302910.021589:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:0.0:1789302910.021618:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850a0f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1651 expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021673:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021686:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021699:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021712:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021718:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021724:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021776:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021784:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021792:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021798:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f121dca00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302910.021802:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021810:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021821:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302910.021824:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302910.021829:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021835:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dca00/0x64fe430951850a16 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021846:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021852:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021858:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021863:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:0.0:1789302910.021865:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021876:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021884:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xbb834312 (ffff960f0cf83c00) 00010000:00010000:0.0:1789302910.021886:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1789302910.021889:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021894:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72000/0x64fe430951850a1d lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xbb834312 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021921:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850a0f lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1651 expref: 111 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021930:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:0.0:1789302910.021933:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1789302910.021936:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121ddc00/0x64fe430951850a0f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1651 expref: 111 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.021952:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f121ddc00, rc -2) 00000100:00100000:0.0:1789302910.022013:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:2345:x1876219350601856:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 463us (761us total) trans 0 rc -2/-2 00000100:00100000:0.0:1789302910.027218:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 95 00000100:00100000:2.0:1789302910.034533:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302910.035708:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350601984 00000100:00100000:0.0:1789302910.035720:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350601984 00000100:00100000:0.0:1789302910.035797:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 1684 00000100:00100000:0.0:1789302910.035807:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+110:33635:x1876219350601984:12345-192.168.203.11@tcp:36:rm.0 00010000:00010000:0.0:1789302910.035859:0:5843:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1789302910.035886:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345b000/0x64fe4309518505e0 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a149f expref: 110 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.035903:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:0.0:1789302910.035907:0:5843:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1789302910.035927:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 3/0,1 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.035940:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.035949:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.035963:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 3/0,1 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.035970:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.035976:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.036068:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.036078:0:5843:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 2/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 52 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.036087:0:5843:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 4/0,1 mode: --/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.036119:0:5843:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 52 pid: 18427 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.036131:0:5843:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 52 pid: 18427 timeout: 847 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302910.036144:0:5843:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371195904:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302910.036230:0:5843:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fda000 going to sleep for 11 seconds 00000100:00100000:0.0:1789302910.056230:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302910.056329:0:5843:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f024a80 pname:cluuid:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000_UUID:5843:1876219371195904:192.168.203.11@tcp:104: 00010000:00010000:1.0:1789302910.056357:0:5843:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302910.056582:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350602112 00000100:00100000:2.0:1789302910.056591:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350602112 00010000:00010000:2.0:1789302910.056645:0:8463:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 53 pid: 18427 timeout: 847 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302910.056675:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00100000:2.0:1789302910.056684:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd06850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+53:2348:x1876219350602112:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:2.0:1789302910.056698:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1789302910.056704:0:8463:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 4/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 53 pid: 18427 timeout: 847 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.056716:0:8463:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 3/0,0 mode: PR/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 53 pid: 18427 timeout: 847 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.056737:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e024b8a00/0x64fe430951850585 lrc: 0/0,0 mode: --/PR res: [0x200000402:0x36:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a141a expref: 53 pid: 18427 timeout: 847 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.056762:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:2.0:1789302910.056767:0:8463:0:(ldlm_inodebits.c:93:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 0 ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,1 mode: --/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.056775:0:8463:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.056783:0:8463:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 4/0,1 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40290000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.056805:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1789302910.056848:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd06850 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350602112:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 164us (626us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302910.056856:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 506 00000100:00080000:1.0:1789302910.058197:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302910.058215:0:5843:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.058226:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:1.0:1789302910.058907:0:5843:0:(llog_osd.c:725:llog_osd_write_rec()) added record [0x1:0x6:0x0].20, 64 off9544 00000004:00080000:1.0:1789302910.058917:0:5843:0:(osp_sync.c:1847:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 747069414654ns, next at 748069412051ns, rc = 0 00010000:00010000:1.0:1789302910.058999:0:5843:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 3/0,1 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT flags: 0x40000000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059012:0:5843:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010080000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059023:0:5843:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 3/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059030:0:5843:0:(ldlm_lock.c:954:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x40010080000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059037:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 4/0,1 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059043:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT flags: 0x50010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059062:0:5843:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302910.059081:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,1 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059087:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 2/0,0 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059093:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059099:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302910.059103:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 3/0,0 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059110:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 4/0,0 mode: EX/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059120:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:1.0:1789302910.059123:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302910.059128:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 2/0,0 mode: --/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059133:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72a00/0x64fe430951850a32 lrc: 0/0,0 mode: --/EX res: [0x200000402:0x36:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059151:0:5843:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 3/0,1 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059157:0:5843:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 2/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059162:0:5843:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059167:0:5843:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72600 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302910.059170:0:5843:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 3/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059176:0:5843:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 4/0,0 mode: CW/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059188:0:5843:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302910.059190:0:5843:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302910.059193:0:5843:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 2/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302910.059199:0:5843:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72600/0x64fe430951850a24 lrc: 0/0,0 mode: --/CW res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000004:00010000:1.0:1789302910.059208:0:5843:0:(mdt_handler.c:4489:mdt_save_lock()) ### save lock request 000000006c72392d reply state 0000000000daaa37 transno 4294968396 ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302910.059215:0:5843:0:(service.c:179:ptlrpc_save_lock()) nlocks 0 00000100:00100000:1.0:1789302910.059301:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+111:33635:x1876219350601984:12345-192.168.203.11@tcp:36:rm.0 Request processed in 23483us (24787us total) trans 4294968396 rc 0/0 00000100:00100000:1.0:1789302910.059309:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 1684 00010000:00010000:0.0:1789302910.059602:0:5832:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1789302910.059609:0:5832:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.059615:0:5832:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a400 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:0.0:1789302910.059618:0:5832:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 2/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40010480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.059625:0:5832:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 3/0,0 mode: PW/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x40018480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.059635:0:5832:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x36:0x0].0x0 (ffff960f2c94d900) 00010000:00010000:0.0:1789302910.059638:0:5832:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1789302910.059641:0:5832:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 1/0,0 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.059646:0:5832:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a400/0x64fe430951850a39 lrc: 0/0,0 mode: --/PW res: [0x200000402:0x36:0x0].0x0 bits 0x40/0x0 rrc: 2 type: IBT flags: 0x44819480000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302910.059661:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302910.059663:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302910.059665:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302910.065469:0:3286:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(TXN) ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 3/0,1 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.065483:0:3286:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 2/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.065491:0:3286:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.065496:0:3286:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72400 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302910.065706:0:3286:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 3/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.065716:0:3286:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 4/0,0 mode: PW/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.065731:0:3286:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x5bff4890 (ffff960f0cf83400) 00010000:00010000:2.0:1789302910.065734:0:3286:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302910.065740:0:3286:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 2/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302910.065746:0:3286:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72400/0x64fe430951850a2b lrc: 0/0,0 mode: --/PW res: [0x200000401:0x53:0x0].0x5bff4890 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5843 initiator: MDT0 lru_score: 0 lru_type: 0 00000040:00080000:0.0:1789302910.156727:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302910.156737:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302910.158587:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 54 count 1 00000040:00080000:0.0:1789302910.158600:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302910.229380:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302910.229421:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302910.229452:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302910.229481:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302910.229540:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350602368 00000100:00100000:2.0:1789302910.229550:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350602368 00000100:00100000:2.0:1789302910.229607:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350602624 00000100:00100000:2.0:1789302910.229611:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350602624 00000100:00100000:2.0:1789302910.229629:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350602496 00000100:00100000:2.0:1789302910.229632:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350602496 00000100:00100000:2.0:1789302910.229656:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 552 00000100:00100000:2.0:1789302910.229666:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350602368:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302910.229734:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350602368:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 69us (311us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302910.229741:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 552 00000100:00100000:0.0:1789302910.229916:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 553 00000100:00100000:0.0:1789302910.229927:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350602624:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302910.229966:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2348:x1876219350602624:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 42us (512us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302910.229972:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 553 00000100:00100000:0.0:1789302910.229989:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 554 00000100:00100000:0.0:1789302910.229995:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350602496:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302910.230018:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2347:x1876219350602496:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (536us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302910.230024:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 554 00000100:00100000:3.0:1789302910.230110:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350602240 00000100:00100000:3.0:1789302910.230116:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350602240 00000100:00100000:3.0:1789302910.230156:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 630 00000100:00100000:3.0:1789302910.230163:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350602240:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302910.230188:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350602240:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (818us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302910.230193:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 630 00000100:00080000:3.0:1789302910.230241:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302910.230245:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302910.247920:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302910.248080:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350602752 00000100:00100000:1.0:1789302910.248091:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350602752 00000100:00100000:1.0:1789302910.248169:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 555 00000100:00100000:1.0:1789302910.248180:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350602752:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302910.248250:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350602752:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 73us (339us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302910.248257:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 555 00000100:00080000:2.0:1789302910.249337:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:0.0:1789302910.605077:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302910.984609:0:23206:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302910.984618:0:23206:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302910.985200:0:23206:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302910.985211:0:23206:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302911.631602:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302912.652730:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302913.152877:0:5865:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-MDT0000: transno 4294968396 is committed 00000004:00080000:2.0:1789302913.152895:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 23 committed 00000004:00080000:2.0:1789302913.152917:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 25 committed 00000004:00080000:2.0:1789302913.152936:0:5865:0:(osp_sync.c:1812:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 24 committed 00000100:00100000:1.0:1789302913.153088:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302913.153115:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371196032:0@lo:6:osp-syn-1-0.0 00000100:00100000:2.0:1789302913.153592:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302913.153613:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371196160:0@lo:6:osp-syn-0-0.0 00000100:00100000:2.0:1789302913.153682:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302913.153895:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302913.155481:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371196160 00000100:00100000:2.0:1789302913.155497:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371196160 00000100:00100000:2.0:1789302913.155599:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371196032 00000100:00100000:2.0:1789302913.155603:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371196032 00000100:00100000:2.0:1789302913.155636:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:2.0:1789302913.155650:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+7:3300:x1876219371196160:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:2.0:1789302913.155691:0:18578:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x240000400:34 count 1 00010000:00010000:2.0:1789302913.155729:0:18578:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302913.155758:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302913.155770:0:18578:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 2/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302913.155779:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=13s v=(0 0) 00010000:00010000:2.0:1789302913.155784:0:18578:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:2.0:1789302913.155789:0:18578:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 5/0,1 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302913.155834:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302913.155850:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 17 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302913.155867:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371196288:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302913.155981:0:18578:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 10 pid: 6697 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302913.155998:0:18578:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 10 pid: 6697 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302913.156009:0:18578:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371196416:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302913.156037:0:18578:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f07e4e480 going to sleep for 11 seconds 00000100:00100000:0.0:1789302913.156143:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:0.0:1789302913.156155:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371196032:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:0.0:1789302913.156170:0:13439:0:(ofd_dev.c:1862:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x280000400:34 count 1 00010000:00010000:0.0:1789302913.156188:0:13439:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302913.156200:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 9 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302913.156211:0:13439:0:(ldlm_lock.c:687:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302913.156217:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=13s v=(0 0) 00010000:00010000:0.0:1789302913.156220:0:13439:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:0.0:1789302913.156224:0:13439:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 5/0,1 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302913.156246:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 17 pid: 6695 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302913.156257:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 17 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302913.156283:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371196544:192.168.203.11@tcp:104: 00010000:00010000:0.0:1789302913.156324:0:13439:0:(ldlm_lockd.c:963:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 9 pid: 18578 timeout: 0 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789302913.156334:0:13439:0:(ldlm_lockd.c:495:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 9 pid: 18578 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:0.0:1789302913.156356:0:13439:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371196672:192.168.203.11@tcp:104: 00000100:00100000:0.0:1789302913.156378:0:13439:0:(client.c:2716:ptlrpc_set_wait()) set ffff960f08fda300 going to sleep for 11 seconds 00000100:00080000:3.0:1789302913.160331:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302913.160354:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000001:00080000:3.0:1789302913.163140:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967392 is committed 00000100:00100000:2.0:1789302913.176464:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7f800 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371196288:192.168.203.11@tcp:104: 00000100:00100000:2.0:1789302913.176483:0:18578:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7ed80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:18578:1876219371196416:192.168.203.11@tcp:104: 00010000:00010000:2.0:1789302913.176543:0:18578:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 3/0,1 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00100000:2.0:1789302913.177393:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302913.177538:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302913.177562:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371196544:192.168.203.11@tcp:104: 00000100:00100000:3.0:1789302913.182710:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603008 00000100:00100000:3.0:1789302913.182729:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603008 00010000:00010000:3.0:1789302913.182804:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302913.182845:0:5829:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603136 00000100:00100000:3.0:1789302913.182849:0:5829:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603136 00010000:00010000:3.0:1789302913.182871:0:5829:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302913.182897:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 507 00000100:00100000:3.0:1789302913.182906:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350603008:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302913.182923:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302913.182929:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.182945:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.182976:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000078d1b910/0x64fe430951850872 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15ef expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.183009:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302913.183142:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7ed80 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350603008:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 236us (5753us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302913.183150:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 507 00000100:00100000:3.0:1789302913.183170:0:5829:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 508 00000100:00100000:3.0:1789302913.183176:0:5829:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+18:2348:x1876219350603136:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:3.0:1789302913.183186:0:5829:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1789302913.183191:0:5829:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.183201:0:5829:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.183217:0:5829:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000004d03796/0x64fe430951850856 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 5 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15e8 expref: 18 pid: 6695 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.183229:0:5829:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1789302913.183247:0:5829:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7f800 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350603136:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 72us (5710us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302913.183253:0:5829:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 508 00000100:00080000:3.0:1789302913.183285:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302913.183289:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789302913.183458:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302913.183587:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302913.184041:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350602880 00000100:00100000:1.0:1789302913.184055:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350602880 00010000:00010000:1.0:1789302913.184114:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 11 pid: 6697 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302913.184145:0:5831:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603264 00000100:00100000:1.0:1789302913.184149:0:5831:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603264 00010000:00010000:1.0:1789302913.184167:0:5831:0:(ldlm_lockd.c:2683:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 10 pid: 18578 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:1.0:1789302913.184184:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 509 00000100:00100000:1.0:1789302913.184193:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+11:2347:x1876219350602880:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302913.184207:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302913.184213:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 4/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 11 pid: 6697 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184225:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 3/0,0 mode: PR/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 11 pid: 6697 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184246:0:5831:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009b7703d8/0x64fe430951850753 lrc: 0/0,0 mode: --/PR res: [0x240000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1563 expref: 11 pid: 6697 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184386:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302913.184392:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x240000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302913.184399:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184409:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 4/0,1 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184436:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302913.184493:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e680 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350602880:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 300us (1036us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302913.184526:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 509 00000100:00100000:1.0:1789302913.184553:0:5831:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00100000:1.0:1789302913.184561:0:5831:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350603264:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789302913.184569:0:5831:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1789302913.184575:0:5831:0:(ldlm_lockd.c:1796:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 10 pid: 18578 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184585:0:5831:0:(ldlm_lockd.c:568:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 10 pid: 18578 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184599:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302913.184602:0:5831:0:(ldlm_extent.c:408:ldlm_update_contention()) add 0 to [0x280000400:0x22:0x0].0x0 since=0s v=(0 0) 00010000:00010000:1.0:1789302913.184664:0:5831:0:(ldlm_lock.c:715:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184672:0:5831:0:(ldlm_resource.c:1750:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184690:0:5831:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789302913.184710:0:5831:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7dc00 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350603264:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 151us (1123us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302913.184716:0:5831:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 510 00000100:00080000:1.0:1789302913.184739:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302913.184743:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789302913.184755:0:18578:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184764:0:18578:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184774:0:18578:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 3/0,1 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184783:0:18578:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.184801:0:18578:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1789302913.185548:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371196160:12345-0@lo:6:osp-syn-0-0.0 Request processed in 29898us (31863us total) trans 4294967393 rc 0/0 00000100:00100000:1.0:1789302913.185558:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00010000:00010000:1.0:1789302913.185585:0:17746:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302913.185592:0:17746:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.185720:0:17746:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302913.185725:0:17746:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 2/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.185734:0:17746:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 3/0,0 mode: PW/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.185749:0:17746:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 1/0,0 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.185756:0:17746:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000797bbe74/0x64fe430951850a40 lrc: 0/0,0 mode: --/PW res: [0x240000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 18578 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.185777:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302913.185779:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302913.185780:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302913.185817:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967392 gen 2 00000004:00080000:1.0:1789302913.185823:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f754e00, transno 4294967391 00000004:00080000:1.0:1789302913.185834:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3f755500, transno 4294967392 00000100:00100000:1.0:1789302913.185841:0:3300:0:(client.c:3170:ptlrpc_free_committed()) @@@ stopping search req@ffff960f3ad45180 x1876219371196160/t4294967393(4294967393) o6->lustre-OST0000-osc-MDT0000@0@lo:28/4 lens 544/432 e 0 to 0 dl 1789302929 ref 2 fl Rpc:RQU/204/0 rc 0/0 job:'osp-syn-0-0.0' uid:0 gid:0 projid:4294967295 00000004:00080000:1.0:1789302913.185851:0:3300:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960f3ad45180/2, rc 0, transno 4294967393 00000100:00100000:1.0:1789302913.185857:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad45180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371196160:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1789302913.187044:0:13439:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000bfff6874/0x64fe4309518507fb lrc: 0/0,0 mode: --/PR res: [0x280000400:0x22:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a15a2 expref: 9 pid: 18578 timeout: 850 lvb_type: 1 lru_score: 0 lru_type: 0 00000100:00100000:3.0:1789302913.187071:0:13439:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f025c00 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:13439:1876219371196672:192.168.203.11@tcp:104: 00010000:00010000:3.0:1789302913.187099:0:13439:0:(ldlm_request.c:298:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.187113:0:13439:0:(ldlm_request.c:193:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.187127:0:13439:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.187140:0:13439:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.187149:0:13439:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302913.187176:0:13439:0:(ldlm_lockd.c:2189:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1789302913.187234:0:5833:0:(ldlm_lockd.c:2801:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1789302913.187244:0:5833:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.187255:0:5833:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f25f72a00 already unused, calling callback (ffffffffc16b18b0) 00010000:00010000:1.0:1789302913.187262:0:5833:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.187286:0:5833:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.187304:0:5833:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.187310:0:5833:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000094d19f9f/0x64fe430951850a47 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x22:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 13439 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302913.187334:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302913.187336:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302913.187337:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1789302913.189021:0:3301:0:(osp_sync.c:622:osp_sync_interpret()) reply req ffff960e02b08380/2, rc 0, transno 4294967391 00000100:00100000:1.0:1789302913.189031:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b08380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371196032:0@lo:6:osp-syn-1-0.0 00000100:00100000:3.0:1789302913.190298:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+6:3301:x1876219371196032:12345-0@lo:6:osp-syn-1-0.0 Request processed in 34135us (36394us total) trans 4294967391 rc 0/0 00000100:00100000:3.0:1789302913.190314:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000040:00080000:2.0:1789302913.212689:0:23261:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302913.212704:0:23261:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302913.215359:0:23261:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302913.215383:0:23261:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302913.676756:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302914.062191:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f754e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371196928:0@lo:400:kworker.0 00000100:00100000:1.0:1789302914.062207:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02b09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371196800:0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.062356:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371197056:0@lo:400:kworker.0 00000100:00100000:3.0:1789302914.066103:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302914.066284:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371197440:0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.066299:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302914.066351:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302914.066361:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302914.066378:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371197184:0@lo:400:kworker.0 00000100:00100000:3.0:1789302914.066416:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371196928 00000100:00100000:0.0:1789302914.066424:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302914.066429:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371196928 00000100:00100000:0.0:1789302914.066474:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197440 00000100:00100000:0.0:1789302914.066484:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197440 00000100:00100000:3.0:1789302914.066541:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197056 00000100:00100000:3.0:1789302914.066548:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197056 00000100:00100000:0.0:1789302914.066566:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197184 00000100:00100000:0.0:1789302914.066573:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197184 00000100:00100000:3.0:1789302914.066603:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 529 00000100:00100000:0.0:1789302914.066614:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 556 00000100:00100000:3.0:1789302914.066620:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371196928:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.066630:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371197440:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.066787:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3303:x1876219371197440:12345-0@lo:400:kworker.0 Request processed in 162us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302914.066795:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 556 00000100:00100000:0.0:1789302914.066815:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 557 00000100:00100000:0.0:1789302914.066822:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371197184:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.066855:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371197184:12345-0@lo:400:kworker.0 Request processed in 34us (433us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302914.066861:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 557 00000100:00100000:0.0:1789302914.066909:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371197184:0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.067043:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:0.0:1789302914.067051:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371197056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.067089:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371197056:12345-0@lo:400:kworker.0 Request processed in 39us (800us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302914.067095:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:0.0:1789302914.067125:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302914.067150:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371197312:0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.067174:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302914.067193:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371196928:12345-0@lo:400:kworker.0 Request processed in 576us (1101us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302914.067207:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 529 00000100:00100000:3.0:1789302914.067230:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197312 00000100:00100000:3.0:1789302914.067236:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197312 00000100:00100000:0.0:1789302914.067289:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371197056:0@lo:400:kworker.0 00000100:00100000:3.0:1789302914.067296:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 531 00000100:00100000:3.0:1789302914.067310:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371197312:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302914.067367:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371197312:12345-0@lo:400:kworker.0 Request processed in 61us (194us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302914.067373:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 531 00000100:00080000:0.0:1789302914.067493:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302914.067605:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371196800 00000100:00100000:0.0:1789302914.067611:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371196800 00000100:00100000:3.0:1789302914.067626:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371197312:0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.067655:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:0.0:1789302914.067666:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371196800:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302914.067722:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371196800:12345-0@lo:400:kworker.0 Request processed in 57us (595us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302914.067728:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00100000:3.0:1789302914.067744:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371197440:0@lo:400:kworker.0 00000100:00100000:3.0:1789302914.067772:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f754e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371196928:0@lo:400:kworker.0 00000100:00100000:3.0:1789302914.067837:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02b09880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371196800:0@lo:400:kworker.0 00000001:00080000:1.0:1789302914.127616:0:7654:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0001: transno 4294967391 is committed 00000100:00100000:2.0:1789302914.573693:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302914.573722:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371197568:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302914.573820:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302914.573946:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197568 00000100:00100000:1.0:1789302914.573957:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197568 00000100:00100000:1.0:1789302914.574017:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:1.0:1789302914.574029:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371197568:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302914.574079:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302914.574143:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371197568:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302914.574151:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:2.0:1789302914.574347:0:3301:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294967391 gen 2 00000004:00080000:2.0:1789302914.574354:0:3301:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960e02b08380, transno 4294967391 00000100:00100000:2.0:1789302914.574379:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371197568:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302914.701602:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302915.090722:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302915.090753:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371197696:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302915.090840:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302915.091290:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197696 00000100:00100000:1.0:1789302915.091301:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197696 00000100:00100000:1.0:1789302915.091366:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 229 00000100:00100000:1.0:1789302915.091377:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371197696:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302915.091421:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302915.091473:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f754e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3300:x1876219371197696:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (632us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302915.091480:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 229 00000100:00100000:0.0:1789302915.091619:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371197696:0@lo:13:osp-pre-0-0.0 00000100:00080000:0.0:1789302915.091657:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:2.0:1789302915.094967:0:23314:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302915.094978:0:23314:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302915.096445:0:23314:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302915.096462:0:23314:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302915.349368:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302915.349420:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302915.349458:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302915.349484:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302915.349543:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302915.349568:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302915.349593:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302915.349681:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603648 00000100:00100000:1.0:1789302915.349694:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603648 00000100:00100000:1.0:1789302915.349781:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604160 00000100:00100000:1.0:1789302915.349785:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604160 00000100:00100000:1.0:1789302915.349806:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603776 00000100:00100000:1.0:1789302915.349809:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603776 00000100:00100000:1.0:1789302915.349822:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604032 00000100:00100000:1.0:1789302915.349824:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604032 00000100:00100000:1.0:1789302915.349844:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 558 00000100:00100000:1.0:1789302915.349856:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350603648:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302915.349946:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350603648:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 91us (524us total) trans 0 rc 0/0 00000100:00080000:2.0:1789302915.350092:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302915.350098:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302915.350142:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 558 00000100:00100000:1.0:1789302915.350162:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 559 00000100:00100000:1.0:1789302915.350169:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350604160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302915.350198:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350604160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (713us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302915.350204:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 559 00000100:00100000:1.0:1789302915.350217:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 560 00000100:00100000:1.0:1789302915.350223:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350603776:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302915.350246:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350603776:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (678us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302915.350262:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 560 00000100:00100000:1.0:1789302915.350275:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 561 00000100:00100000:1.0:1789302915.350282:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350604032:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302915.350307:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350604032:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (713us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302915.350313:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 561 00000100:00100000:3.0:1789302915.350659:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603904 00000100:00100000:3.0:1789302915.350666:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603904 00000100:00100000:3.0:1789302915.350731:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603520 00000100:00100000:3.0:1789302915.350735:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603520 00000100:00100000:3.0:1789302915.350762:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 532 00000100:00100000:3.0:1789302915.350769:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350603904:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302915.350812:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350603904:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 44us (1353us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302915.350819:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 532 00000100:00100000:3.0:1789302915.350833:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00100000:3.0:1789302915.350841:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350603520:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302915.350875:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350603520:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (1332us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302915.350881:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 533 00000100:00080000:2.0:1789302915.351060:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302915.351132:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350603392 00000100:00100000:0.0:1789302915.351138:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350603392 00000100:00100000:0.0:1789302915.351179:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 632 00000100:00100000:0.0:1789302915.351186:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350603392:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302915.351217:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350603392:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (1865us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302915.351224:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 632 00010000:00010000:1.0:1789302915.725236:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302916.748706:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302917.013122:0:23365:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302917.013135:0:23365:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302917.013705:0:23365:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302917.013719:0:23365:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302917.772874:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1789302918.208099:0:6719:0:(tgt_lastrcvd.c:983:tgt_cb_last_committed()) lustre-OST0000: transno 4294967393 is committed 00010000:00010000:1.0:1789302918.797675:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302919.037151:0:23418:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302919.037165:0:23418:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302919.039306:0:23418:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302919.039328:0:23418:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302919.181167:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371198080:0@lo:400:kworker.0 00000100:00100000:0.0:1789302919.181270:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302919.181883:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198080 00000100:00100000:0.0:1789302919.181894:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198080 00000100:00100000:0.0:1789302919.182052:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:0.0:1789302919.182064:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371198080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302919.182129:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371198080:12345-0@lo:400:kworker.0 Request processed in 68us (859us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302919.182137:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00100000:0.0:1789302919.182187:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371198080:0@lo:400:kworker.0 00000100:00100000:0.0:1789302919.182281:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302919.182290:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371198208:0@lo:400:kworker.0 00000100:00100000:0.0:1789302919.182321:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302919.182791:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371197952:0@lo:400:kworker.0 00000100:00100000:3.0:1789302919.182832:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1789302919.182895:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302919.182972:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1789302919.182980:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371197824:0@lo:400:kworker.0 00000100:00100000:3.0:1789302919.183012:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302919.183920:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197824 00000100:00100000:2.0:1789302919.183933:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197824 00000100:00100000:2.0:1789302919.184002:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:2.0:1789302919.184013:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371197824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302919.184086:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371197824:12345-0@lo:400:kworker.0 Request processed in 76us (1072us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302919.184093:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:1.0:1789302919.184394:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198208 00000100:00100000:1.0:1789302919.184400:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198208 00000100:00100000:1.0:1789302919.184442:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00100000:1.0:1789302919.184450:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371198208:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789302919.184523:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789302919.184738:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371197824:0@lo:400:kworker.0 00000100:00100000:1.0:1789302919.185780:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371198208:12345-0@lo:400:kworker.0 Request processed in 1326us (3452us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302919.185796:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00100000:2.0:1789302919.185875:0:3300:0:(client.c:3126:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294967393 gen 2 00000004:00080000:2.0:1789302919.185882:0:3300:0:(osp_sync.c:570:osp_sync_request_commit_cb()) commit req ffff960f3ad45180, transno 4294967393 00000100:00100000:2.0:1789302919.185902:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371198208:0@lo:400:kworker.0 00000100:00100000:0.0:1789302919.186085:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371197952 00000100:00100000:0.0:1789302919.186094:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371197952 00000100:00100000:0.0:1789302919.186144:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 535 00000100:00100000:0.0:1789302919.186152:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371197952:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302919.186211:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371197952:12345-0@lo:400:kworker.0 Request processed in 60us (3378us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302919.186217:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 535 00000100:00100000:2.0:1789302919.186287:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371198336:0@lo:400:kworker.0 00000100:00100000:2.0:1789302919.186332:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302919.186363:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1789302919.186363:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198336 00000100:00100000:1.0:1789302919.186367:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198336 00000100:00100000:2.0:1789302919.186369:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371198464:0@lo:400:kworker.0 00000100:00100000:2.0:1789302919.186393:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302919.186439:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:1.0:1789302919.186446:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371198336:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302919.186479:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371198336:12345-0@lo:400:kworker.0 Request processed in 34us (148us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302919.186485:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00080000:1.0:1789302919.186604:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302919.186754:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371197952:0@lo:400:kworker.0 00000100:00100000:2.0:1789302919.186811:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371198336:0@lo:400:kworker.0 00000100:00100000:2.0:1789302919.186826:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198464 00000100:00100000:2.0:1789302919.186829:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198464 00000100:00080000:1.0:1789302919.186865:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302919.186909:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 563 00000100:00100000:2.0:1789302919.186917:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371198464:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302919.186960:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371198464:12345-0@lo:400:kworker.0 Request processed in 44us (566us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302919.186966:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 563 00000100:00100000:3.0:1789302919.186984:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371198464:0@lo:400:kworker.0 00000100:00100000:2.0:1789302919.693179:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371198592:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302919.693286:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302919.693542:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198592 00000100:00100000:1.0:1789302919.693553:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198592 00000100:00100000:1.0:1789302919.693617:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1789302919.693630:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371198592:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302919.693677:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302919.693792:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371198592:12345-0@lo:13:osp-pre-1-0.0 Request processed in 164us (504us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302919.693800:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:0.0:1789302919.702880:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371198592:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302919.821220:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302920.210348:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371198720:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302920.210471:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302920.210692:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198720 00000100:00100000:1.0:1789302920.210703:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198720 00000100:00100000:1.0:1789302920.210780:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 231 00000100:00100000:1.0:1789302920.210800:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371198720:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302920.210848:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302920.210908:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371198720:12345-0@lo:13:osp-pre-0-0.0 Request processed in 111us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302920.210916:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 231 00000100:00100000:1.0:1789302920.210971:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371198720:0@lo:13:osp-pre-0-0.0 00000040:00080000:0.0:1789302920.397193:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302920.397209:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302920.397912:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302920.397928:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302920.468986:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302920.469035:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302920.469075:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302920.469103:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302920.469127:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302920.469146:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302920.469203:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302920.469246:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604672 00000100:00100000:1.0:1789302920.469258:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604672 00000100:00100000:1.0:1789302920.469334:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604544 00000100:00100000:1.0:1789302920.469338:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604544 00000100:00100000:1.0:1789302920.469357:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604928 00000100:00100000:1.0:1789302920.469360:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604928 00000100:00100000:1.0:1789302920.469373:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605056 00000100:00100000:1.0:1789302920.469376:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605056 00000100:00100000:1.0:1789302920.469395:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 564 00000100:00100000:1.0:1789302920.469406:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350604672:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302920.469496:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350604672:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 91us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302920.469533:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 564 00000100:00100000:1.0:1789302920.469555:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 565 00000100:00100000:1.0:1789302920.469562:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350604544:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302920.469592:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350604544:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (489us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302920.469597:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 565 00000100:00100000:1.0:1789302920.469608:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 566 00000100:00100000:1.0:1789302920.469615:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350604928:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302920.469638:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350604928:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (510us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302920.469643:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 566 00000100:00100000:1.0:1789302920.469654:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:1.0:1789302920.469660:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350605056:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302920.469683:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350605056:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (478us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302920.469689:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:1.0:1789302920.469719:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604416 00000100:00100000:1.0:1789302920.469723:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604416 00000100:00100000:1.0:1789302920.469754:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604800 00000100:00100000:1.0:1789302920.469757:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604800 00000100:00100000:1.0:1789302920.469779:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 537 00000100:00100000:1.0:1789302920.469785:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350604416:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302920.469817:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350604416:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (782us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302920.469824:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 537 00000100:00100000:1.0:1789302920.469836:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 538 00000100:00100000:1.0:1789302920.469842:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350604800:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302920.469865:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350604800:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (718us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302920.469871:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 538 00000100:00080000:1.0:1789302920.469891:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302920.469898:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302920.469900:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302920.473072:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350604288 00000100:00100000:3.0:1789302920.473094:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350604288 00000100:00100000:3.0:1789302920.473183:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 634 00000100:00100000:3.0:1789302920.473195:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350604288:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302920.473314:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350604288:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 121us (4343us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302920.473321:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 634 00010000:00010000:3.0:1789302920.845662:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302921.057868:0:23469:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302921.057881:0:23469:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302921.058420:0:23469:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302921.058433:0:23469:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302921.868732:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302922.892717:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302923.273046:0:23521:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302923.273060:0:23521:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302923.274096:0:23521:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302923.274114:0:23521:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302923.918766:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302924.300939:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371199232:0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.301412:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302924.301498:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302924.301527:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371199104:0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.301560:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302924.301611:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371199232 00000100:00100000:1.0:1789302924.301620:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371199232 00000100:00100000:1.0:1789302924.301813:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 568 00000100:00100000:1.0:1789302924.301825:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371199232:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.301904:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371199232:12345-0@lo:400:kworker.0 Request processed in 81us (869us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302924.301913:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 568 00000100:00100000:1.0:1789302924.302178:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371199104 00000100:00100000:1.0:1789302924.302183:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371199104 00000100:00100000:1.0:1789302924.302222:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 539 00000100:00100000:1.0:1789302924.302229:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371199104:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.302262:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371199104:12345-0@lo:400:kworker.0 Request processed in 33us (701us total) trans 0 rc 0/0 00000100:00080000:0.0:1789302924.302467:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302924.302473:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302924.302525:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371199104:0@lo:400:kworker.0 00000100:00100000:0.0:1789302924.302538:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbc700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371199232:0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.302674:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 539 00000100:00100000:3.0:1789302924.302835:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371198848:0@lo:400:kworker.0 00000100:00100000:3.0:1789302924.302894:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302924.303101:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371199360:0@lo:400:kworker.0 00000100:00100000:3.0:1789302924.303131:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302924.303162:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1789302924.303169:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371198976:0@lo:400:kworker.0 00000100:00100000:3.0:1789302924.303203:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302924.303226:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371199488:0@lo:400:kworker.0 00000100:00100000:3.0:1789302924.303253:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302924.303355:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371199360 00000100:00100000:1.0:1789302924.303363:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371199360 00000100:00100000:1.0:1789302924.303404:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198976 00000100:00100000:1.0:1789302924.303407:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198976 00000100:00100000:1.0:1789302924.303428:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 540 00000100:00100000:1.0:1789302924.303436:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371199360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.303482:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad45180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371199360:12345-0@lo:400:kworker.0 Request processed in 47us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302924.303489:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 540 00000100:00100000:1.0:1789302924.303519:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 541 00000100:00100000:1.0:1789302924.303525:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371198976:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.303554:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371198976:12345-0@lo:400:kworker.0 Request processed in 29us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302924.303560:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 541 00000100:00100000:0.0:1789302924.303739:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371198848 00000100:00100000:0.0:1789302924.303744:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371198848 00000100:00100000:1.0:1789302924.303795:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371199488 00000100:00100000:1.0:1789302924.303799:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371199488 00000100:00100000:1.0:1789302924.303834:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 569 00000100:00100000:1.0:1789302924.303841:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f08598450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371199488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302924.303887:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f08598450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371199488:12345-0@lo:400:kworker.0 Request processed in 47us (634us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302924.303893:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 569 00000100:00080000:1.0:1789302924.304181:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302924.304186:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302924.312456:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371199488:0@lo:400:kworker.0 00000100:00100000:3.0:1789302924.312484:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbc380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371199360:0@lo:400:kworker.0 00000100:00100000:3.0:1789302924.312543:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371198976:0@lo:400:kworker.0 00000100:00100000:0.0:1789302924.312797:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:0.0:1789302924.312822:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371198848:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302924.313148:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371198848:12345-0@lo:400:kworker.0 Request processed in 339us (10251us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302924.313157:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:2.0:1789302924.313389:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371198848:0@lo:400:kworker.0 00000100:00100000:0.0:1789302924.814973:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302924.815006:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371199616:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302924.815107:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302924.817271:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371199616 00000100:00100000:2.0:1789302924.817291:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371199616 00000100:00100000:2.0:1789302924.817373:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1789302924.817385:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371199616:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302924.817435:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302924.817588:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371199616:12345-0@lo:13:osp-pre-1-0.0 Request processed in 205us (2480us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302924.817595:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:2.0:1789302924.817662:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371199616:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302924.940789:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302925.326595:0:23575:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302925.326608:0:23575:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302925.327282:0:23575:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302925.327297:0:23575:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302925.339408:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371199744:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302925.339555:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302925.342052:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371199744 00000100:00100000:1.0:1789302925.342073:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371199744 00000100:00100000:1.0:1789302925.342151:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 233 00000100:00100000:1.0:1789302925.342163:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371199744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302925.342228:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302925.342347:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371199744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 186us (2793us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302925.342356:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 233 00000100:00100000:2.0:1789302925.342742:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371199744:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302925.588353:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302925.588494:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302925.589611:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302925.589810:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302925.589845:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302925.589877:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302925.589919:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302925.589968:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605440 00000100:00100000:3.0:1789302925.589980:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605440 00000100:00100000:3.0:1789302925.590058:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605952 00000100:00100000:3.0:1789302925.590062:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605952 00000100:00100000:3.0:1789302925.590092:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605568 00000100:00100000:3.0:1789302925.590095:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605568 00000100:00100000:3.0:1789302925.590120:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605824 00000100:00100000:3.0:1789302925.590123:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605824 00000100:00100000:3.0:1789302925.590150:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 570 00000100:00100000:3.0:1789302925.590164:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350605440:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302925.591772:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605184 00000100:00100000:2.0:1789302925.591793:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605184 00000100:00100000:3.0:1789302925.591850:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350605440:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1683us (3349us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302925.591861:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 636 00000100:00100000:3.0:1789302925.591866:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 570 00000100:00100000:2.0:1789302925.591874:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350605184:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302925.591897:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 571 00000100:00100000:3.0:1789302925.591908:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350605952:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302925.591924:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350605184:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 54us (4692us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302925.591932:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 636 00000100:00100000:3.0:1789302925.591946:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350605952:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 40us (2138us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302925.591954:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 571 00000100:00100000:3.0:1789302925.591972:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 572 00000100:00100000:3.0:1789302925.591980:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350605568:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302925.592008:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350605568:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (2133us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302925.592014:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 572 00000100:00100000:3.0:1789302925.592030:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:3.0:1789302925.592039:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350605824:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302925.592067:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350605824:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (2150us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302925.592073:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:3.0:1789302925.592096:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605696 00000100:00100000:3.0:1789302925.592101:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605696 00000100:00100000:3.0:1789302925.592142:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 542 00000100:00100000:3.0:1789302925.592150:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350605696:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302925.592175:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350605312 00000100:00100000:0.0:1789302925.592194:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350605312 00000100:00100000:3.0:1789302925.592195:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b08a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350605696:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (2579us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302925.592204:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 542 00000100:00100000:0.0:1789302925.592277:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 543 00000100:00100000:0.0:1789302925.592286:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350605312:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302925.592316:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350605312:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (2473us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302925.592324:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 543 00000100:00080000:3.0:1789302925.593126:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302925.593133:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302925.593136:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:2.0:1789302925.967130:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302926.995694:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302927.502948:0:23626:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302927.502956:0:23626:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302927.503481:0:23626:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302927.503491:0:23626:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302928.012746:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302929.036677:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302929.163598:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302929.163733:0:5849:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606080 00000100:00100000:2.0:1789302929.163744:0:5849:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606080 00000100:00100000:2.0:1789302929.163810:0:5849:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 96 00000100:00100000:2.0:1789302929.163827:0:5849:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350606080:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:2.0:1789302929.163860:0:5849:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:2.0:1789302929.163881:0:5849:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850a4e lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1682 expref: -99 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.163936:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.163955:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.163965:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.163978:0:5849:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.163985:0:5849:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00000100:00080000:3.0:1789302929.164088:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789302929.164206:0:5849:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165370:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165383:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165389:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165394:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01516000 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302929.165398:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165409:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165424:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302929.165426:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302929.165431:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165437:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01516000/0x64fe430951850a55 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165451:0:5849:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165457:0:5849:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165462:0:5849:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165467:0:5849:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f01517c00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:2.0:1789302929.165469:0:5849:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165475:0:5849:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165481:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x70954b0b (ffff960f095b2d00) 00010000:00010000:2.0:1789302929.165483:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1789302929.165486:0:5849:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165495:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517c00/0x64fe430951850a5c lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0x70954b0b bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5849 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165564:0:5849:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850a4e lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1682 expref: 109 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165574:0:5849:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:2.0:1789302929.165576:0:5849:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1789302929.165579:0:5849:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01517e00/0x64fe430951850a4e lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1682 expref: 109 pid: 5849 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:2.0:1789302929.165588:0:5849:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f01517e00, rc -2) 00000100:00100000:2.0:1789302929.165723:0:5849:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350606080:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 1897us (2133us total) trans 0 rc -2/-2 00000100:00100000:2.0:1789302929.165732:0:5849:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 96 00000100:00100000:3.0:1789302929.169891:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302929.169995:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606208 00000100:00100000:3.0:1789302929.170006:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606208 00000100:00100000:3.0:1789302929.170062:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 97 00000100:00100000:3.0:1789302929.170071:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350606208:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:3.0:1789302929.170103:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:3.0:1789302929.170121:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850a63 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1689 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00000100:00080000:2.0:1789302929.170393:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789302929.171442:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.171459:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.171471:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.171486:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.171494:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.171514:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172307:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172317:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172324:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172330:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960e01bf2a00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302929.172334:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172342:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172355:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302929.172358:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302929.172362:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172368:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf2a00/0x64fe430951850a6a lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172378:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172383:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172389:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172395:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f07ef7200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:3.0:1789302929.172397:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172402:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172408:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xae1c5c33 (ffff960f0cdf2100) 00010000:00010000:3.0:1789302929.172410:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1789302929.172412:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172417:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7200/0x64fe430951850a71 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172453:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850a63 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1689 expref: 53 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172463:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:3.0:1789302929.172465:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1789302929.172469:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e01bf3c00/0x64fe430951850a63 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1689 expref: 53 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:3.0:1789302929.172478:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960e01bf3c00, rc -2) 00000100:00100000:3.0:1789302929.172591:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350606208:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 2519us (2706us total) trans 0 rc -2/-2 00000100:00100000:3.0:1789302929.172601:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 97 00000040:00080000:2.0:1789302929.336934:0:23680:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302929.336944:0:23680:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302929.338118:0:23680:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302929.338130:0:23680:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302929.431415:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371199872:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.431525:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302929.431572:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371200384:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.431598:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302929.431622:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789302929.431627:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371200000:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.431648:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302929.431666:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371200512:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.431686:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302929.431723:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371200128:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.431746:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302929.431763:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302929.431768:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371200256:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.431785:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302929.431824:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371199872 00000100:00100000:2.0:1789302929.431832:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371199872 00000100:00100000:2.0:1789302929.431875:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:2.0:1789302929.431883:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371199872:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.431929:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371199872:12345-0@lo:400:kworker.0 Request processed in 46us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302929.431935:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:2.0:1789302929.431964:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200384 00000100:00100000:2.0:1789302929.431968:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200384 00000100:00100000:2.0:1789302929.431998:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200000 00000100:00100000:2.0:1789302929.432001:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200000 00000100:00100000:2.0:1789302929.432018:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200128 00000100:00100000:2.0:1789302929.432020:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200128 00000100:00100000:2.0:1789302929.432035:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 544 00000100:00100000:2.0:1789302929.432041:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371200384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432073:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371200384:12345-0@lo:400:kworker.0 Request processed in 31us (474us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302929.432079:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 544 00000100:00100000:2.0:1789302929.432091:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 545 00000100:00100000:2.0:1789302929.432097:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371200000:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432124:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371200000:12345-0@lo:400:kworker.0 Request processed in 27us (474us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302929.432129:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 545 00000100:00100000:2.0:1789302929.432141:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:2.0:1789302929.432146:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371200128:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432188:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371200128:12345-0@lo:400:kworker.0 Request processed in 41us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302929.432193:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:2.0:1789302929.432212:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200512 00000100:00100000:2.0:1789302929.432216:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200512 00000100:00100000:2.0:1789302929.432241:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200256 00000100:00100000:2.0:1789302929.432245:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200256 00000100:00100000:2.0:1789302929.432265:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:2.0:1789302929.432272:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371200512:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432302:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371200512:12345-0@lo:400:kworker.0 Request processed in 31us (615us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302929.432309:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00000100:00100000:2.0:1789302929.432321:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:2.0:1789302929.432327:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371200256:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432357:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371200256:12345-0@lo:400:kworker.0 Request processed in 30us (570us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302929.432363:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:2.0:1789302929.432405:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371200000:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432415:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef6a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371200512:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432425:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371199872:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432434:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371200384:0@lo:400:kworker.0 00000100:00080000:2.0:1789302929.432458:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302929.432462:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302929.432464:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302929.432484:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371200256:0@lo:400:kworker.0 00000100:00100000:2.0:1789302929.432492:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371200128:0@lo:400:kworker.0 00000100:00100000:1.0:1789302929.938712:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302929.938735:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371200640:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302929.938800:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302929.939048:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200640 00000100:00100000:0.0:1789302929.939056:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200640 00000100:00100000:0.0:1789302929.939109:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:0.0:1789302929.939119:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371200640:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302929.939163:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302929.939221:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371200640:12345-0@lo:13:osp-pre-1-0.0 Request processed in 103us (420us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302929.939398:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371200640:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302929.939459:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00010000:00010000:0.0:1789302930.060713:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302930.449362:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371200768:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302930.449438:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302930.449839:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200768 00000100:00100000:0.0:1789302930.449848:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200768 00000100:00100000:0.0:1789302930.449905:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 235 00000100:00100000:0.0:1789302930.449915:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371200768:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302930.449951:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302930.450007:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371200768:12345-0@lo:13:osp-pre-0-0.0 Request processed in 94us (569us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302930.450012:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 235 00000100:00100000:2.0:1789302930.450447:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371200768:0@lo:13:osp-pre-0-0.0 00000040:00080000:1.0:1789302930.637263:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302930.637272:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302930.637942:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302930.637954:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789302930.709037:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302930.709423:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302930.709456:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302930.709475:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302930.709490:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302930.709527:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302930.709551:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302930.709580:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606336 00000100:00100000:3.0:1789302930.709590:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606336 00000100:00100000:3.0:1789302930.709634:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 638 00000100:00100000:3.0:1789302930.709643:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350606336:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302930.709706:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350606336:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 64us (679us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302930.709713:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 638 00000100:00100000:3.0:1789302930.709741:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606592 00000100:00100000:3.0:1789302930.709745:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606592 00000100:00100000:3.0:1789302930.709775:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350607104 00000100:00100000:3.0:1789302930.709778:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350607104 00000100:00100000:3.0:1789302930.709795:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606720 00000100:00100000:3.0:1789302930.709797:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606720 00000100:00100000:3.0:1789302930.709808:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606976 00000100:00100000:3.0:1789302930.709810:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606976 00000100:00100000:3.0:1789302930.709823:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 576 00000100:00100000:3.0:1789302930.709829:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350606592:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302930.709856:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350606592:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (781us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302930.709863:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 576 00000100:00100000:3.0:1789302930.709875:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:3.0:1789302930.709881:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350607104:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302930.709902:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350607104:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (426us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302930.709908:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:3.0:1789302930.709918:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 578 00000100:00100000:3.0:1789302930.709929:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350606720:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302930.709948:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350606720:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302930.709955:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 578 00000100:00100000:3.0:1789302930.709965:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 579 00000100:00100000:3.0:1789302930.709971:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350606976:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302930.709990:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350606976:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 19us (439us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302930.709997:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 579 00000100:00100000:3.0:1789302930.710013:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606848 00000100:00100000:3.0:1789302930.710016:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606848 00000100:00100000:3.0:1789302930.710038:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350606464 00000100:00100000:3.0:1789302930.710040:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350606464 00000100:00100000:3.0:1789302930.710056:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 547 00000100:00100000:3.0:1789302930.710062:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350606848:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302930.710251:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350606848:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 188us (793us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302930.710257:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 547 00000100:00100000:3.0:1789302930.710270:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 548 00000100:00100000:3.0:1789302930.710573:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350606464:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302930.710602:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350606464:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 327us (1112us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302930.710609:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 548 00000100:00080000:3.0:1789302930.710754:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302930.710758:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789302931.097255:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302931.500908:0:23731:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302931.500916:0:23731:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302931.515605:0:23731:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302931.515631:0:23731:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302932.110539:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302933.146820:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302933.753694:0:23782:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302933.753703:0:23782:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302933.757419:0:23782:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302933.757439:0:23782:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302934.156820:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302934.541456:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371201152:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.541545:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302934.541626:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302934.541633:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371201280:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.541659:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302934.541724:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201152 00000100:00100000:1.0:1789302934.541730:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201152 00000100:00100000:1.0:1789302934.541795:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 549 00000100:00100000:1.0:1789302934.541806:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371201152:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.541851:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371201152:12345-0@lo:400:kworker.0 Request processed in 48us (306us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302934.541858:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 549 00000100:00100000:1.0:1789302934.541883:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371201024:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.541905:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302934.541923:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371201536:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.541942:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302934.541961:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789302934.541966:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371200896:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.541989:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302934.542008:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371201408:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.542026:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302934.542222:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201024 00000100:00100000:0.0:1789302934.542227:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201024 00000100:00100000:0.0:1789302934.542251:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201408 00000100:00100000:0.0:1789302934.542255:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201408 00000100:00100000:0.0:1789302934.542273:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 550 00000100:00100000:0.0:1789302934.542280:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371201024:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302934.542315:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371201024:12345-0@lo:400:kworker.0 Request processed in 36us (408us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302934.542321:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 550 00000100:00100000:0.0:1789302934.542333:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 551 00000100:00100000:0.0:1789302934.542339:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371201408:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302934.542362:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371201408:12345-0@lo:400:kworker.0 Request processed in 23us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302934.542367:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 551 00000100:00080000:0.0:1789302934.542381:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302934.542412:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371201152:0@lo:400:kworker.0 00000100:00100000:0.0:1789302934.542441:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201280 00000100:00100000:0.0:1789302934.542445:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201280 00000100:00100000:0.0:1789302934.542472:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201536 00000100:00100000:0.0:1789302934.542474:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201536 00000100:00100000:0.0:1789302934.542492:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 580 00000100:00100000:0.0:1789302934.542497:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371201280:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302934.542552:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371201280:12345-0@lo:400:kworker.0 Request processed in 54us (892us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302934.542558:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 580 00000100:00100000:0.0:1789302934.542568:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 581 00000100:00100000:0.0:1789302934.542574:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371201536:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302934.542598:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371201536:12345-0@lo:400:kworker.0 Request processed in 24us (655us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302934.542606:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 581 00000100:00080000:0.0:1789302934.542627:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302934.542630:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302934.542646:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371201280:0@lo:400:kworker.0 00000100:00100000:0.0:1789302934.542679:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371200896 00000100:00100000:0.0:1789302934.542682:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371200896 00000100:00100000:0.0:1789302934.542704:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 639 00000100:00100000:0.0:1789302934.542709:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371200896:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302934.542749:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371200896:12345-0@lo:400:kworker.0 Request processed in 28us (747us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302934.542755:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 639 00000100:00100000:1.0:1789302934.542905:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371201408:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.542915:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371201024:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.542924:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371201536:0@lo:400:kworker.0 00000100:00100000:1.0:1789302934.542937:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371200896:0@lo:400:kworker.0 00000100:00100000:0.0:1789302935.053741:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302935.053830:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371201664:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302935.053919:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302935.054003:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201664 00000100:00100000:0.0:1789302935.054012:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201664 00000100:00100000:0.0:1789302935.054068:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:0.0:1789302935.054079:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371201664:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302935.056609:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302935.056774:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371201664:12345-0@lo:13:osp-pre-1-0.0 Request processed in 2683us (2841us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302935.056788:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:1.0:1789302935.056912:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f7c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371201664:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302935.181956:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302935.572317:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302935.572359:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02322680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371201792:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302935.572449:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302935.572546:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201792 00000100:00100000:0.0:1789302935.572557:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201792 00000100:00100000:0.0:1789302935.572617:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789302935.572628:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371201792:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302935.572672:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302935.572733:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371201792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302935.572741:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 237 00000100:00100000:0.0:1789302935.572806:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02322680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371201792:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302935.825264:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302935.825355:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350607232 00000100:00100000:2.0:1789302935.825370:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350607232 00000100:00100000:2.0:1789302935.825435:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 640 00000100:00100000:2.0:1789302935.825446:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350607232:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302935.825567:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350607232:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 123us (319us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302935.825576:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 640 00000100:00100000:2.0:1789302935.827869:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302935.827945:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350607360 00000100:00100000:0.0:1789302935.827959:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350607360 00000100:00100000:0.0:1789302935.828056:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 552 00000100:00100000:0.0:1789302935.828069:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350607360:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302935.828172:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350607360:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 105us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302935.828180:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 552 00000100:00080000:2.0:1789302935.829081:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302935.832615:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302935.832648:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302935.832674:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302935.832708:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350607488 00000100:00100000:2.0:1789302935.832720:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350607488 00000100:00100000:2.0:1789302935.832788:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608000 00000100:00100000:2.0:1789302935.832791:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608000 00000100:00100000:2.0:1789302935.832814:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 582 00000100:00100000:2.0:1789302935.832826:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350607488:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302935.832915:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350607488:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 92us (313us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302935.832923:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 582 00000100:00100000:2.0:1789302935.832941:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00100000:2.0:1789302935.832948:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350608000:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302935.832974:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350608000:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302935.832980:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00080000:2.0:1789302935.832997:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302935.833001:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302935.833135:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350607744 00000100:00100000:0.0:1789302935.833140:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350607744 00000100:00100000:0.0:1789302935.833193:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 553 00000100:00100000:0.0:1789302935.833202:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350607744:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302935.833235:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350607744:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302935.833241:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 553 00000100:00080000:0.0:1789302935.833272:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302935.834696:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302935.834737:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302935.834806:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350607616 00000100:00100000:3.0:1789302935.834816:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350607616 00000100:00100000:3.0:1789302935.834874:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350607872 00000100:00100000:3.0:1789302935.834877:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350607872 00000100:00100000:3.0:1789302935.834898:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 584 00000100:00100000:3.0:1789302935.834907:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350607616:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302935.834976:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350607616:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 70us (287us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302935.834984:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 584 00000100:00100000:3.0:1789302935.835002:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 585 00000100:00100000:3.0:1789302935.835009:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350607872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302935.835169:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350607872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 159us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302935.835176:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 585 00000100:00080000:0.0:1789302935.835289:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302935.835295:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000040:00080000:3.0:1789302935.952053:0:23834:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302935.952069:0:23834:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302935.952771:0:23834:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302935.952785:0:23834:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302936.206724:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302937.228637:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302937.882818:0:23885:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302937.882988:0:23885:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302937.884352:0:23885:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302937.884366:0:23885:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302938.253460:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302939.276795:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302939.670122:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371201920:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.670358:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302939.670404:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0114b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371202432:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.670432:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302939.670459:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789302939.670465:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371202048:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.670485:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302939.670522:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01149f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371202560:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.670547:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302939.670575:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371202176:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.670601:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302939.670618:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302939.670624:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371202304:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.670644:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302939.670671:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371201920 00000100:00100000:2.0:1789302939.670679:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371201920 00000100:00100000:2.0:1789302939.670722:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:2.0:1789302939.670730:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371201920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.671073:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202432 00000100:00100000:3.0:1789302939.671078:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202432 00000100:00100000:3.0:1789302939.671128:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202048 00000100:00100000:3.0:1789302939.671131:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202048 00000100:00100000:3.0:1789302939.671148:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202176 00000100:00100000:3.0:1789302939.671150:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202176 00000100:00100000:3.0:1789302939.671178:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 554 00000100:00100000:3.0:1789302939.671185:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371202432:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.671243:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114bb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371202432:12345-0@lo:400:kworker.0 Request processed in 58us (810us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302939.671250:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 554 00000100:00100000:3.0:1789302939.671267:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 555 00000100:00100000:3.0:1789302939.671273:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371202048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.671299:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371202048:12345-0@lo:400:kworker.0 Request processed in 28us (814us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302939.671305:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 555 00000100:00100000:3.0:1789302939.671316:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 556 00000100:00100000:3.0:1789302939.671321:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371202176:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.671349:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371202176:12345-0@lo:400:kworker.0 Request processed in 29us (748us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302939.671354:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 556 00000100:00100000:3.0:1789302939.671373:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202560 00000100:00100000:3.0:1789302939.671377:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202560 00000100:00100000:3.0:1789302939.671401:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202304 00000100:00100000:3.0:1789302939.671404:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202304 00000100:00100000:3.0:1789302939.671423:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1789302939.671429:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371202560:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.671460:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371202560:12345-0@lo:400:kworker.0 Request processed in 32us (913us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302939.671466:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1789302939.671478:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 587 00000100:00100000:3.0:1789302939.671484:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371202304:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.671531:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371202304:12345-0@lo:400:kworker.0 Request processed in 48us (886us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302939.671537:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 587 00000100:00080000:3.0:1789302939.671585:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302939.671589:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302939.671591:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302939.671735:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371202048:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.672269:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371201920:12345-0@lo:400:kworker.0 Request processed in 1539us (1910us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302939.672277:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:2.0:1789302939.672325:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371202304:0@lo:400:kworker.0 00000100:00100000:2.0:1789302939.672337:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371202176:0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.672408:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01149f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371202560:0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.672418:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371201920:0@lo:400:kworker.0 00000100:00100000:3.0:1789302939.672427:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0114b480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371202432:0@lo:400:kworker.0 00000040:00080000:3.0:1789302939.854819:0:23936:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302939.854828:0:23936:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302939.862997:0:23936:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302939.863013:0:23936:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302940.173257:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789302940.173280:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371202688:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789302940.173345:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302940.173565:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202688 00000100:00100000:0.0:1789302940.173574:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202688 00000100:00100000:0.0:1789302940.173630:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:0.0:1789302940.173640:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371202688:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302940.173677:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302940.173734:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371202688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (388us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302940.173741:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:3.0:1789302940.173826:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371202688:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302940.300979:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302940.428604:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302940.428880:0:5848:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608128 00000100:00100000:1.0:1789302940.428906:0:5848:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608128 00000100:00100000:1.0:1789302940.429038:0:5848:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 98 00000100:00100000:1.0:1789302940.429050:0:5848:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f07735050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350608128:12345-192.168.203.11@tcp:63:ll_sa.0 00010000:00010000:1.0:1789302940.429111:0:5848:0:(ldlm_lockd.c:1265:ldlm_handle_enqueue()) ### server-side enqueue handler START 00010000:00010000:1.0:1789302940.429138:0:5848:0:(ldlm_lockd.c:1343:ldlm_handle_enqueue()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe430951850a78 lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xed5c2244f73a1690 expref: -99 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430157:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 3/1,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430186:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430202:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430221:0:5848:0:(ldlm_lock.c:775:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 3/1,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x0/0x0 rrc: 2 type: IBT flags: 0x40000000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430229:0:5848:0:(ldlm_lock.c:1072:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430235:0:5848:0:(ldlm_request.c:557:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430497:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 3/1,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430526:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 2/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430534:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430540:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345ac00 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302940.430545:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 3/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430558:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 4/0,0 mode: CR/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430572:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302940.430575:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302940.430581:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 2/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430588:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345ac00/0x64fe430951850a7f lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430602:0:5848:0:(ldlm_lock.c:829:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 3/1,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210000000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430609:0:5848:0:(ldlm_lock.c:887:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 2/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x50210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430615:0:5848:0:(ldlm_lockd.c:1907:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430621:0:5848:0:(ldlm_lockd.c:1921:ldlm_handle_bl_callback()) Lock ffff960f3345a200 already unused, calling callback (ffffffffc20d94f0) 00010000:00010000:1.0:1789302940.430624:0:5848:0:(ldlm_request.c:378:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 3/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40210400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430630:0:5848:0:(ldlm_request.c:1416:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 4/0,0 mode: PR/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x40218400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430637:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0xae1c5c33 (ffff960f2c94d100) 00010000:00010000:1.0:1789302940.430640:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1789302940.430643:0:5848:0:(ldlm_lockd.c:1933:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 2/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430653:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345a200/0x64fe430951850a86 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x53:0x0].0xae1c5c33 bits 0x2/0x0 rrc: 2 type: IBT flags: 0x44a19400000000 pid: 5848 initiator: MDT0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430693:0:5848:0:(ldlm_lockd.c:1513:ldlm_handle_enqueue()) ### server-side enqueue handler, sending reply (err=-2, rc=-2) ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe430951850a78 lrc: 1/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1690 expref: 109 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430704:0:5848:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x53:0x0].0x0 (ffff960f124a9f00) 00010000:00010000:1.0:1789302940.430707:0:5848:0:(ldlm_inodebits.c:76:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:1.0:1789302940.430711:0:5848:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3345be00/0x64fe430951850a78 lrc: 0/0,0 mode: --/CR res: [0x200000401:0x53:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f73a1690 expref: 109 pid: 5848 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789302940.430723:0:5848:0:(ldlm_lockd.c:1592:ldlm_handle_enqueue()) ### server-side enqueue handler END (lock ffff960f3345be00, rc -2) 00000100:00100000:1.0:1789302940.430840:0:5848:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f07735050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350608128:12345-192.168.203.11@tcp:63:ll_sa.0 Request processed in 1791us (2247us total) trans 0 rc -2/-2 00000100:00100000:1.0:1789302940.430850:0:5848:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 98 00000100:00080000:3.0:1789302940.435910:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302940.685114:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371202816:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302940.685227:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302940.685453:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202816 00000100:00100000:1.0:1789302940.685464:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202816 00000100:00100000:1.0:1789302940.685555:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 239 00000100:00100000:1.0:1789302940.685567:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371202816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302940.685613:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302940.685675:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371202816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 110us (448us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302940.685682:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 239 00000100:00100000:3.0:1789302940.686482:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371202816:0@lo:13:osp-pre-0-0.0 00000040:00080000:1.0:1789302940.877449:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302940.877463:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302940.878543:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302940.878564:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302940.948062:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302940.948122:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302940.948157:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302940.948190:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302940.948214:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302940.948231:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302940.948251:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302940.948290:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608512 00000100:00100000:2.0:1789302940.948312:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608512 00000100:00100000:2.0:1789302940.948382:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609024 00000100:00100000:2.0:1789302940.948385:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609024 00000100:00100000:2.0:1789302940.948403:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608640 00000100:00100000:2.0:1789302940.948405:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608640 00000100:00100000:2.0:1789302940.948417:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608896 00000100:00100000:2.0:1789302940.948420:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608896 00000100:00100000:2.0:1789302940.948438:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00100000:2.0:1789302940.948449:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350608512:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302940.948560:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350608512:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 112us (437us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302940.948568:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00100000:2.0:1789302940.948587:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 589 00000100:00100000:2.0:1789302940.948594:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350609024:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302940.948619:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350609024:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302940.948625:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 589 00000100:00100000:2.0:1789302940.948636:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 590 00000100:00100000:2.0:1789302940.948641:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350608640:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302940.948660:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350608640:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (429us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302940.948666:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 590 00000100:00100000:2.0:1789302940.948673:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 591 00000100:00100000:2.0:1789302940.948678:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350608896:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302940.948695:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350608896:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 17us (444us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302940.948700:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 591 00000100:00100000:1.0:1789302940.949317:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608768 00000100:00100000:1.0:1789302940.949329:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608768 00000100:00100000:1.0:1789302940.949386:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608384 00000100:00100000:1.0:1789302940.949390:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608384 00000100:00100000:1.0:1789302940.949416:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 557 00000100:00100000:1.0:1789302940.949425:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350608768:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302940.949485:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350608768:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 62us (1326us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302940.949492:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 557 00000100:00100000:1.0:1789302940.949540:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 558 00000100:00100000:1.0:1789302940.949548:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350608384:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302940.949600:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350608384:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 53us (1385us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302940.949607:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 558 00000100:00100000:0.0:1789302940.950042:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350608256 00000100:00100000:0.0:1789302940.950048:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350608256 00000100:00100000:0.0:1789302940.950091:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00100000:0.0:1789302940.950099:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350608256:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302940.950146:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350608256:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 48us (2098us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302940.950153:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00080000:3.0:1789302940.950944:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302940.950953:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302940.950956:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789302941.325449:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302941.957629:0:23987:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302941.957642:0:23987:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302941.958439:0:23987:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302941.958456:0:23987:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302942.348846:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302943.373011:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302944.204451:0:24039:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302944.204464:0:24039:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302944.206282:0:24039:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302944.206301:0:24039:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302944.402142:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302944.784448:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371203200:0@lo:400:kworker.0 00000100:00100000:0.0:1789302944.784541:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302944.784621:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302944.784628:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371203328:0@lo:400:kworker.0 00000100:00100000:0.0:1789302944.784656:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302944.784844:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371203072:0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.784874:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302944.784891:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371203584:0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.784909:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302944.784925:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1789302944.784931:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371202944:0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.784961:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302944.784982:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371203456:0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.785004:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302944.785031:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203328 00000100:00100000:2.0:1789302944.785039:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203328 00000100:00100000:2.0:1789302944.785095:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203584 00000100:00100000:2.0:1789302944.785098:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203584 00000100:00100000:2.0:1789302944.785119:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 592 00000100:00100000:2.0:1789302944.785127:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371203328:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.785188:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371203328:12345-0@lo:400:kworker.0 Request processed in 60us (530us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302944.785195:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 592 00000100:00100000:2.0:1789302944.785211:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 593 00000100:00100000:2.0:1789302944.785217:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371203584:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.785248:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371203584:12345-0@lo:400:kworker.0 Request processed in 32us (338us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302944.785254:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 593 00000100:00100000:2.0:1789302944.785271:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203200 00000100:00100000:2.0:1789302944.785274:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203200 00000100:00100000:2.0:1789302944.785297:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203072 00000100:00100000:2.0:1789302944.785300:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203072 00000100:00100000:2.0:1789302944.785312:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203456 00000100:00100000:2.0:1789302944.785315:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203456 00000100:00100000:2.0:1789302944.785328:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 559 00000100:00100000:2.0:1789302944.785333:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371203200:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.785362:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371203200:12345-0@lo:400:kworker.0 Request processed in 28us (822us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302944.785368:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 559 00000100:00100000:2.0:1789302944.785378:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 560 00000100:00100000:2.0:1789302944.785383:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371203072:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302944.785825:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371203328:0@lo:400:kworker.0 00000100:00100000:0.0:1789302944.785838:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371203200:0@lo:400:kworker.0 00000100:00100000:0.0:1789302944.785890:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371203584:0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.786000:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371203072:12345-0@lo:400:kworker.0 Request processed in 616us (1124us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302944.786007:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 560 00000100:00100000:2.0:1789302944.786022:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 561 00000100:00100000:2.0:1789302944.786030:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371203456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302944.786071:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371203456:12345-0@lo:400:kworker.0 Request processed in 43us (1066us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302944.786076:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 561 00000100:00100000:3.0:1789302944.786162:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371202944 00000100:00100000:3.0:1789302944.786167:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371202944 00000100:00100000:3.0:1789302944.786215:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 643 00000100:00100000:3.0:1789302944.786223:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371202944:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302944.786255:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371202944:12345-0@lo:400:kworker.0 Request processed in 33us (1294us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302944.786261:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 643 00000100:00080000:3.0:1789302944.786286:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302944.786291:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789302944.786293:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302944.786545:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371202944:0@lo:400:kworker.0 00000100:00100000:0.0:1789302944.786569:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371203456:0@lo:400:kworker.0 00000100:00100000:0.0:1789302944.786578:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371203072:0@lo:400:kworker.0 00000100:00100000:2.0:1789302945.292925:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371203712:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302945.293002:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302945.293158:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203712 00000100:00100000:0.0:1789302945.293167:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203712 00000100:00100000:0.0:1789302945.293217:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:0.0:1789302945.293228:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371203712:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302945.293263:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302945.293329:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371203712:12345-0@lo:13:osp-pre-1-0.0 Request processed in 103us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302945.293336:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:2.0:1789302945.293545:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371203712:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302945.433803:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302945.812682:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302945.812715:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371203840:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302945.812794:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302945.812872:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203840 00000100:00100000:1.0:1789302945.812880:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203840 00000100:00100000:1.0:1789302945.812928:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 241 00000100:00100000:1.0:1789302945.812937:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371203840:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302945.812977:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302945.813025:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371203840:12345-0@lo:13:osp-pre-0-0.0 Request processed in 89us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302945.813032:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 241 00000100:00100000:1.0:1789302945.813098:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371203840:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302946.074778:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302946.074838:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302946.074872:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302946.074894:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302946.074911:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302946.074931:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302946.074955:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302946.074982:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609408 00000100:00100000:0.0:1789302946.074991:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609408 00000100:00100000:0.0:1789302946.075060:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609536 00000100:00100000:0.0:1789302946.075063:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609536 00000100:00100000:0.0:1789302946.075081:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609792 00000100:00100000:0.0:1789302946.075084:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609792 00000100:00100000:0.0:1789302946.075097:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609920 00000100:00100000:0.0:1789302946.075099:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609920 00000100:00100000:0.0:1789302946.075116:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 594 00000100:00100000:0.0:1789302946.075125:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350609408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302946.075188:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350609408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 63us (314us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302946.075195:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 594 00000100:00100000:0.0:1789302946.075212:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 595 00000100:00100000:0.0:1789302946.075218:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350609536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302946.075241:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350609536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302946.075247:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 595 00000100:00100000:0.0:1789302946.075259:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 596 00000100:00100000:0.0:1789302946.075265:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350609792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302946.075285:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350609792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (354us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302946.075291:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 596 00000100:00100000:0.0:1789302946.075302:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 597 00000100:00100000:0.0:1789302946.075308:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350609920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302946.075328:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350609920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (373us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302946.075334:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 597 00000100:00100000:0.0:1789302946.075354:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609280 00000100:00100000:0.0:1789302946.075357:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609280 00000100:00100000:0.0:1789302946.075384:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609664 00000100:00100000:0.0:1789302946.075387:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609664 00000100:00100000:0.0:1789302946.075406:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 562 00000100:00100000:0.0:1789302946.075412:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350609280:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302946.075438:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350609280:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (601us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302946.075445:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 562 00000100:00100000:0.0:1789302946.075456:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 563 00000100:00100000:0.0:1789302946.075462:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350609664:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302946.075485:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02320e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350609664:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (590us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302946.075490:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 563 00000100:00100000:1.0:1789302946.075694:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350609152 00000100:00100000:1.0:1789302946.075699:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350609152 00000100:00100000:1.0:1789302946.075731:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 644 00000100:00100000:1.0:1789302946.075738:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350609152:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302946.075766:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350609152:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (998us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302946.075772:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 644 00000100:00080000:0.0:1789302946.076078:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302946.076083:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302946.076085:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:0.0:1789302946.307778:0:24090:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302946.307787:0:24090:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302946.309232:0:24090:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302946.309246:0:24090:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302946.451840:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302947.478416:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302948.147717:0:24141:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302948.147727:0:24141:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302948.148294:0:24141:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302948.148303:0:24141:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302948.492795:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302949.516633:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302949.905342:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371203968:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905412:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302949.905455:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371204480:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905480:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302949.905527:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789302949.905534:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371204096:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905556:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302949.905573:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371204608:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905594:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302949.905620:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371204224:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905641:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302949.905657:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302949.905662:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371204352:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905679:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302949.905706:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371203968 00000100:00100000:2.0:1789302949.905713:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371203968 00000100:00100000:2.0:1789302949.905751:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:2.0:1789302949.905759:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371203968:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905807:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371203968:12345-0@lo:400:kworker.0 Request processed in 48us (395us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302949.905814:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:2.0:1789302949.905840:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204480 00000100:00100000:2.0:1789302949.905844:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204480 00000100:00100000:2.0:1789302949.905871:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204096 00000100:00100000:2.0:1789302949.905874:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204096 00000100:00100000:2.0:1789302949.905887:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204224 00000100:00100000:2.0:1789302949.905889:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204224 00000100:00100000:2.0:1789302949.905901:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 564 00000100:00100000:2.0:1789302949.905907:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371204480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905935:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371204480:12345-0@lo:400:kworker.0 Request processed in 28us (455us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302949.905940:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 564 00000100:00100000:2.0:1789302949.905950:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 565 00000100:00100000:2.0:1789302949.905955:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371204096:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.905976:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371204096:12345-0@lo:400:kworker.0 Request processed in 22us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302949.905981:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 565 00000100:00100000:2.0:1789302949.905988:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 566 00000100:00100000:2.0:1789302949.905992:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371204224:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906027:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371204224:12345-0@lo:400:kworker.0 Request processed in 33us (384us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302949.906032:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 566 00000100:00100000:2.0:1789302949.906050:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204608 00000100:00100000:2.0:1789302949.906053:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204608 00000100:00100000:2.0:1789302949.906074:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204352 00000100:00100000:2.0:1789302949.906076:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204352 00000100:00100000:2.0:1789302949.906091:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 598 00000100:00100000:2.0:1789302949.906095:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371204608:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906119:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371204608:12345-0@lo:400:kworker.0 Request processed in 24us (525us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302949.906124:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 598 00000100:00100000:2.0:1789302949.906133:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 599 00000100:00100000:2.0:1789302949.906137:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371204352:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906157:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371204352:12345-0@lo:400:kworker.0 Request processed in 20us (477us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302949.906162:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 599 00000100:00080000:2.0:1789302949.906184:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302949.906187:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302949.906190:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302949.906226:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371204352:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906236:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371204224:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906261:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371204096:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906270:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120cb480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371204608:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906277:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371203968:0@lo:400:kworker.0 00000100:00100000:2.0:1789302949.906285:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371204480:0@lo:400:kworker.0 00000040:00080000:0.0:1789302950.094072:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302950.094079:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302950.094867:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302950.094879:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789302950.129649:0:24193:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302950.129659:0:24193:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302950.130186:0:24193:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302950.130196:0:24193:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302950.412873:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371204736:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302950.412940:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302950.413299:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204736 00000100:00100000:3.0:1789302950.413308:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204736 00000100:00100000:3.0:1789302950.413367:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:3.0:1789302950.413376:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371204736:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302950.413410:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302950.413466:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371204736:12345-0@lo:13:osp-pre-1-0.0 Request processed in 91us (525us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302950.413473:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:2.0:1789302950.413694:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371204736:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302950.546092:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302950.924757:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371204864:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302950.924825:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302950.925002:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204864 00000100:00100000:0.0:1789302950.925020:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204864 00000100:00100000:0.0:1789302950.925077:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 243 00000100:00100000:0.0:1789302950.925095:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371204864:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302950.925128:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302950.926394:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371204864:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1295us (1563us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302950.926398:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371204864:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302950.926404:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 243 00000100:00100000:1.0:1789302951.180640:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302951.180766:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610048 00000100:00100000:3.0:1789302951.180779:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610048 00000100:00100000:3.0:1789302951.180860:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 646 00000100:00100000:3.0:1789302951.180872:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350610048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302951.180975:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350610048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 104us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302951.180984:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 646 00000100:00100000:1.0:1789302951.184670:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302951.184771:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302951.184797:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302951.184824:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302951.184883:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610432 00000100:00100000:0.0:1789302951.184896:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610432 00000100:00100000:0.0:1789302951.184966:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610688 00000100:00100000:0.0:1789302951.184969:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610688 00000100:00100000:0.0:1789302951.184991:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 600 00000100:00100000:0.0:1789302951.185013:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350610432:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302951.185103:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350610432:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 103us (330us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302951.185110:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 600 00000100:00100000:0.0:1789302951.185127:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 601 00000100:00100000:0.0:1789302951.185133:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350610688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302951.185155:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350610688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302951.185161:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 601 00000100:00080000:0.0:1789302951.185477:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302951.185483:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302951.186017:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302951.186092:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610816 00000100:00100000:3.0:1789302951.186102:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610816 00000100:00100000:2.0:1789302951.186107:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610176 00000100:00100000:2.0:1789302951.186115:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610176 00000100:00100000:3.0:1789302951.186204:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 602 00000100:00100000:3.0:1789302951.186216:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350610816:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302951.186290:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350610816:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 76us (285us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302951.186296:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 602 00000100:00080000:1.0:1789302951.186488:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302951.186492:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302951.186873:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610560 00000100:00100000:2.0:1789302951.186879:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610560 00000100:00080000:3.0:1789302951.186939:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302951.187015:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:1.0:1789302951.187027:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350610176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302951.187133:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350610176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 109us (2477us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302951.187142:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 567 00000100:00100000:1.0:1789302951.187158:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 568 00000100:00100000:1.0:1789302951.187165:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350610560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302951.187191:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350610560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (2366us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302951.187199:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 568 00000100:00100000:1.0:1789302951.189075:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302951.189212:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610304 00000100:00100000:0.0:1789302951.189223:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610304 00000100:00100000:0.0:1789302951.189401:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 603 00000100:00100000:0.0:1789302951.189413:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350610304:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302951.189532:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350610304:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 92us (438us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302951.189540:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 603 00000100:00080000:3.0:1789302951.191250:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:1.0:1789302951.564874:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302952.015935:0:24244:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302952.015944:0:24244:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302952.022748:0:24244:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302952.022769:0:24244:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302952.591314:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302953.613019:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302953.864414:0:24295:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302953.864423:0:24295:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302953.865081:0:24295:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302953.865091:0:24295:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302954.636664:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302955.020826:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371205120:0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.020933:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302955.020998:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371204992:0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.021014:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371205632:0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.021291:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302955.021350:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371205120 00000100:00100000:0.0:1789302955.021358:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371205120 00000100:00100000:0.0:1789302955.021419:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 569 00000100:00100000:0.0:1789302955.021432:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371205120:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.021497:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371205120:12345-0@lo:400:kworker.0 Request processed in 68us (564us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302955.021527:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 569 00000100:00100000:0.0:1789302955.021580:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371205120:0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.021617:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371205248:0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.021644:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302955.021664:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302955.021670:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371205376:0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.021692:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302955.022043:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371205632 00000100:00100000:3.0:1789302955.022157:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371205632 00000100:00100000:3.0:1789302955.022199:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371205376 00000100:00100000:3.0:1789302955.022203:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371205376 00000100:00100000:3.0:1789302955.022222:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 604 00000100:00100000:3.0:1789302955.022229:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371205632:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302955.022277:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371205632:12345-0@lo:400:kworker.0 Request processed in 49us (989us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302955.022286:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302955.022288:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 604 00000100:00100000:3.0:1789302955.022305:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:3.0:1789302955.022312:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371205376:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302955.022332:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371205504:0@lo:400:kworker.0 00000100:00100000:3.0:1789302955.022348:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371205376:12345-0@lo:400:kworker.0 Request processed in 37us (654us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302955.022358:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00100000:2.0:1789302955.022373:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1789302955.022381:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302955.022388:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302955.022407:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371205248 00000100:00100000:3.0:1789302955.022411:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371205248 00000100:00100000:2.0:1789302955.022422:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371205504 00000100:00100000:2.0:1789302955.022429:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371205504 00000100:00100000:3.0:1789302955.022448:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 570 00000100:00100000:3.0:1789302955.022457:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371205248:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302955.022471:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 571 00000100:00100000:2.0:1789302955.022483:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371205504:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302955.022496:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371205248:12345-0@lo:400:kworker.0 Request processed in 40us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302955.022524:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 570 00000100:00080000:3.0:1789302955.022544:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302955.022551:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120cad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371205504:12345-0@lo:400:kworker.0 Request processed in 69us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302955.022559:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 571 00000100:00100000:2.0:1789302955.022581:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371204992 00000100:00100000:2.0:1789302955.022586:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371204992 00000100:00100000:3.0:1789302955.023150:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371205376:0@lo:400:kworker.0 00000100:00100000:3.0:1789302955.023162:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371205248:0@lo:400:kworker.0 00000100:00100000:3.0:1789302955.023188:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:3.0:1789302955.023196:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371204992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302955.023234:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371204992:12345-0@lo:400:kworker.0 Request processed in 40us (952us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302955.023241:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:2.0:1789302955.023254:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371204992:0@lo:400:kworker.0 00000100:00100000:2.0:1789302955.023271:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db45500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371205504:0@lo:400:kworker.0 00000100:00100000:2.0:1789302955.023314:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db44000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371205632:0@lo:400:kworker.0 00000100:00100000:0.0:1789302955.533754:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302955.533777:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371205760:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302955.533845:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302955.533996:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371205760 00000100:00100000:3.0:1789302955.534004:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371205760 00000100:00100000:3.0:1789302955.534057:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:3.0:1789302955.534067:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371205760:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302955.534103:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302955.534158:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371205760:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (312us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302955.534168:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:1.0:1789302955.534259:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db47100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371205760:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302955.660662:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302955.823933:0:24346:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302955.823942:0:24346:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302955.829223:0:24346:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302955.829243:0:24346:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302956.047786:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371205888:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302956.047855:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302956.047957:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371205888 00000100:00100000:1.0:1789302956.047965:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371205888 00000100:00100000:1.0:1789302956.048022:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 245 00000100:00100000:1.0:1789302956.048033:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371205888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789302956.048066:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789302956.048112:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371205888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (257us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302956.048120:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 245 00000100:00100000:0.0:1789302956.048197:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371205888:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302956.301647:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302956.301691:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302956.301721:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302956.301761:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302956.301797:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611328 00000100:00100000:3.0:1789302956.301808:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611328 00000100:00100000:2.0:1789302956.301848:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611584 00000100:00100000:2.0:1789302956.301854:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611584 00000100:00100000:0.0:1789302956.301913:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611072 00000100:00100000:0.0:1789302956.301918:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611072 00000100:00100000:3.0:1789302956.302649:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 606 00000100:00100000:3.0:1789302956.302661:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350611328:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302956.302728:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350611328:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 69us (1035us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302956.302735:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 606 00000100:00100000:3.0:1789302956.302871:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350610944 00000100:00100000:3.0:1789302956.302877:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350610944 00000100:00100000:3.0:1789302956.302903:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:3.0:1789302956.302910:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350610944:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302956.302940:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350610944:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (1179us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302956.302947:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:3.0:1789302956.304489:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302956.304546:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302956.304566:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302956.304588:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611200 00000100:00100000:3.0:1789302956.304595:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611200 00000100:00100000:2.0:1789302956.304854:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611712 00000100:00100000:2.0:1789302956.304859:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611712 00000100:00100000:2.0:1789302956.304884:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 607 00000100:00100000:2.0:1789302956.304894:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350611584:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302956.304953:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350611584:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 61us (3231us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302956.304962:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 607 00000100:00100000:2.0:1789302956.304988:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:2.0:1789302956.304996:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350611712:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302956.305021:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350611712:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302956.305027:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:2.0:1789302956.305175:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611456 00000100:00100000:2.0:1789302956.305180:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611456 00000100:00100000:2.0:1789302956.305212:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 572 00000100:00100000:2.0:1789302956.305218:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350611456:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302956.305359:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:0.0:1789302956.305368:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350611072:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302956.305396:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350611072:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (3759us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302956.305403:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 573 00000100:00100000:2.0:1789302956.305704:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6afb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350611456:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 485us (1156us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302956.305712:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 572 00000100:00080000:1.0:1789302956.307093:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302956.307109:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302956.307112:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302956.307587:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 609 00000100:00100000:3.0:1789302956.307598:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350611200:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302956.307681:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350611200:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 84us (3197us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302956.307689:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 609 00010000:00010000:3.0:1789302956.685204:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302957.708800:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302957.892280:0:24398:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302957.892288:0:24398:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302957.893027:0:24398:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302957.893037:0:24398:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302958.733582:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789302959.712913:0:24449:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789302959.712923:0:24449:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789302959.715304:0:24449:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789302959.715321:0:24449:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302959.757011:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302960.144384:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371206016:0@lo:400:kworker.0 00000100:00100000:0.0:1789302960.144456:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302960.144530:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371206144:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.144567:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302960.144612:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371206656:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.144638:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302960.144671:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371206272:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.144694:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302960.144714:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302960.144720:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371206400:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.144739:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302960.144770:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206144 00000100:00100000:2.0:1789302960.144777:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206144 00000100:00100000:2.0:1789302960.144818:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206272 00000100:00100000:2.0:1789302960.144821:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206272 00000100:00100000:2.0:1789302960.144862:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:2.0:1789302960.144871:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371206144:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.144920:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ed80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371206144:12345-0@lo:400:kworker.0 Request processed in 49us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302960.144927:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00000100:00100000:2.0:1789302960.144956:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:2.0:1789302960.144964:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371206272:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.144993:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371206272:12345-0@lo:400:kworker.0 Request processed in 30us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302960.144998:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:2.0:1789302960.145014:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206656 00000100:00100000:2.0:1789302960.145017:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206656 00000100:00100000:2.0:1789302960.145040:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206400 00000100:00100000:2.0:1789302960.145042:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206400 00000100:00100000:2.0:1789302960.145059:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:2.0:1789302960.145065:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371206656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.145093:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371206656:12345-0@lo:400:kworker.0 Request processed in 28us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302960.145100:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:2.0:1789302960.145111:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 611 00000100:00100000:2.0:1789302960.145116:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371206400:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.145141:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371206400:12345-0@lo:400:kworker.0 Request processed in 25us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302960.145147:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 611 00000100:00080000:2.0:1789302960.145170:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302960.145174:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302960.145176:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789302960.145212:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371206400:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.145221:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371206272:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.145250:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371206144:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.145257:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371206656:0@lo:400:kworker.0 00000100:00100000:0.0:1789302960.150249:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371206528:0@lo:400:kworker.0 00000100:00100000:3.0:1789302960.150258:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206016 00000100:00100000:3.0:1789302960.150268:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206016 00000100:00100000:3.0:1789302960.150338:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:3.0:1789302960.150348:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371206016:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302960.150418:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371206016:12345-0@lo:400:kworker.0 Request processed in 71us (5961us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302960.150425:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:0.0:1789302960.150476:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302960.150542:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371206016:0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.150624:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206528 00000100:00100000:2.0:1789302960.150629:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206528 00000100:00100000:2.0:1789302960.150675:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 576 00000100:00100000:2.0:1789302960.150682:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371206528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302960.150729:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371206528:12345-0@lo:400:kworker.0 Request processed in 47us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302960.150736:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 576 00000100:00080000:1.0:1789302960.150967:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302960.151401:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371206528:0@lo:400:kworker.0 00000040:00080000:1.0:1789302960.332765:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302960.332777:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302960.333330:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302960.333342:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302960.652992:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371206784:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302960.653064:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302960.653453:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206784 00000100:00100000:1.0:1789302960.653464:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206784 00000100:00100000:1.0:1789302960.653578:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:1.0:1789302960.653591:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371206784:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789302960.653632:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789302960.653705:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371206784:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (639us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302960.653712:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00100000:2.0:1789302960.653805:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371206784:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302960.780685:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302961.164629:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789302961.164648:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371206912:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302961.164706:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302961.164761:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371206912 00000100:00100000:0.0:1789302961.164768:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371206912 00000100:00100000:0.0:1789302961.164814:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 247 00000100:00100000:0.0:1789302961.164822:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371206912:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302961.164854:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302961.164907:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db47100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371206912:12345-0@lo:13:osp-pre-0-0.0 Request processed in 86us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.164914:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 247 00000100:00100000:0.0:1789302961.164961:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371206912:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302961.428536:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302961.428578:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302961.428604:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302961.428624:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302961.428643:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302961.428662:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302961.428685:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302961.428714:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611840 00000100:00100000:0.0:1789302961.428724:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611840 00000100:00100000:0.0:1789302961.428770:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 650 00000100:00100000:0.0:1789302961.428780:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350611840:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302961.428846:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350611840:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 67us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.428854:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 650 00000100:00100000:0.0:1789302961.428887:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612096 00000100:00100000:0.0:1789302961.428891:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612096 00000100:00100000:0.0:1789302961.428929:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612608 00000100:00100000:0.0:1789302961.428943:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612608 00000100:00100000:0.0:1789302961.428965:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612224 00000100:00100000:0.0:1789302961.428967:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612224 00000100:00100000:0.0:1789302961.428989:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612480 00000100:00100000:0.0:1789302961.428993:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612480 00000100:00100000:0.0:1789302961.429008:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:0.0:1789302961.429014:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350612096:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302961.429046:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350612096:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (468us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.429053:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:0.0:1789302961.429065:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 613 00000100:00100000:0.0:1789302961.429071:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350612608:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302961.429093:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350612608:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.429100:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 613 00000100:00100000:0.0:1789302961.429111:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 614 00000100:00100000:0.0:1789302961.429117:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350612224:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302961.429138:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350612224:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (475us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.429143:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 614 00000100:00100000:0.0:1789302961.429153:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 615 00000100:00100000:0.0:1789302961.429158:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350612480:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302961.429180:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350612480:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (494us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.429186:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 615 00000100:00100000:0.0:1789302961.429205:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612352 00000100:00100000:0.0:1789302961.429208:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612352 00000100:00100000:0.0:1789302961.429234:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350611968 00000100:00100000:0.0:1789302961.429237:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350611968 00000100:00100000:0.0:1789302961.429256:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:0.0:1789302961.429263:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350612352:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302961.429290:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350612352:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (685us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.429295:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 577 00000100:00100000:0.0:1789302961.429307:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 578 00000100:00100000:0.0:1789302961.429313:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350611968:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302961.429339:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350611968:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (695us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302961.429345:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 578 00000100:00080000:1.0:1789302961.429968:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302961.429974:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302961.429988:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:3.0:1789302961.516447:0:24500:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302961.516459:0:24500:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302961.517216:0:24500:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302961.517230:0:24500:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302961.804898:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302962.832122:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302963.472456:0:24551:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302963.472465:0:24551:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302963.473051:0:24551:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302963.473062:0:24551:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302963.852678:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302964.876676:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302965.260760:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371207168:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.260836:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302965.260887:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3fe32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371207680:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.260922:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302965.260950:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1789302965.260956:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371207040:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.260982:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302965.261003:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371207552:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261023:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302965.261048:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371207296:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261394:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302965.261431:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789302965.261437:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371207424:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261463:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302965.261497:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207168 00000100:00100000:2.0:1789302965.261524:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207168 00000100:00100000:2.0:1789302965.261566:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207552 00000100:00100000:2.0:1789302965.261569:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207552 00000100:00100000:2.0:1789302965.261593:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207296 00000100:00100000:2.0:1789302965.261597:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207296 00000100:00100000:2.0:1789302965.261614:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 579 00000100:00100000:2.0:1789302965.261622:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371207168:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261673:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcde300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371207168:12345-0@lo:400:kworker.0 Request processed in 51us (836us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302965.261681:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 579 00000100:00100000:2.0:1789302965.261698:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 580 00000100:00100000:2.0:1789302965.261705:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371207552:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261730:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371207552:12345-0@lo:400:kworker.0 Request processed in 27us (706us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302965.261736:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 580 00000100:00100000:2.0:1789302965.261747:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 581 00000100:00100000:2.0:1789302965.261753:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371207296:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261783:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371207296:12345-0@lo:400:kworker.0 Request processed in 30us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302965.261789:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 581 00000100:00100000:2.0:1789302965.261807:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207680 00000100:00100000:2.0:1789302965.261811:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207680 00000100:00100000:2.0:1789302965.261838:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207424 00000100:00100000:2.0:1789302965.261841:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207424 00000100:00100000:2.0:1789302965.261860:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:2.0:1789302965.261866:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371207680:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261895:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371207680:12345-0@lo:400:kworker.0 Request processed in 29us (973us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302965.261900:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00100000:2.0:1789302965.261921:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:2.0:1789302965.261927:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371207424:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.261953:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371207424:12345-0@lo:400:kworker.0 Request processed in 27us (490us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302965.261958:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00100000:2.0:1789302965.262011:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371207424:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.262020:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371207296:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.262047:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef4000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371207552:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.262057:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03ef7100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371207168:0@lo:400:kworker.0 00000100:00100000:2.0:1789302965.262266:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3fe32680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371207680:0@lo:400:kworker.0 00000100:00100000:1.0:1789302965.264025:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207040 00000100:00100000:1.0:1789302965.264038:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207040 00000100:00100000:1.0:1789302965.264090:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:1.0:1789302965.264098:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371207040:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302965.264168:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371207040:12345-0@lo:400:kworker.0 Request processed in 72us (3186us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302965.264175:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00080000:1.0:1789302965.264210:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302965.264215:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302965.264218:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302965.264247:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef33b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371207040:0@lo:400:kworker.0 00000040:00080000:0.0:1789302965.330866:0:24603:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302965.330876:0:24603:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302965.331465:0:24603:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302965.331477:0:24603:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302965.772885:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371207808:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302965.772987:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302965.773164:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207808 00000100:00100000:0.0:1789302965.773178:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207808 00000100:00100000:0.0:1789302965.773226:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:0.0:1789302965.773236:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371207808:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302965.773270:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302965.773326:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371207808:12345-0@lo:13:osp-pre-1-0.0 Request processed in 92us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302965.773333:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:2.0:1789302965.773552:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371207808:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302965.900724:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302966.284763:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302966.284785:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371207936:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302966.284850:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302966.285033:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371207936 00000100:00100000:0.0:1789302966.285041:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371207936 00000100:00100000:0.0:1789302966.285089:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 249 00000100:00100000:0.0:1789302966.285097:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371207936:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302966.285131:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302966.285174:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371207936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 78us (324us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302966.285181:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 249 00000100:00100000:1.0:1789302966.285258:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371207936:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302966.546097:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302966.546138:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302966.546163:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302966.546184:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302966.546201:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302966.546219:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302966.546241:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302966.546266:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350613248 00000100:00100000:3.0:1789302966.546277:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350613248 00000100:00100000:3.0:1789302966.546333:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612864 00000100:00100000:3.0:1789302966.546336:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612864 00000100:00100000:3.0:1789302966.546360:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 582 00000100:00100000:3.0:1789302966.546369:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350613248:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302966.546436:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350613248:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (271us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302966.546443:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 582 00000100:00100000:3.0:1789302966.546459:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00100000:3.0:1789302966.546467:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350612864:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302966.546491:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350612864:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302966.546498:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 583 00000100:00100000:3.0:1789302966.546547:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612992 00000100:00100000:3.0:1789302966.546551:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612992 00000100:00100000:3.0:1789302966.546575:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350613504 00000100:00100000:3.0:1789302966.546578:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350613504 00000100:00100000:3.0:1789302966.546596:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350613120 00000100:00100000:3.0:1789302966.546599:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350613120 00000100:00100000:3.0:1789302966.546612:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350613376 00000100:00100000:3.0:1789302966.546614:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350613376 00000100:00100000:3.0:1789302966.546628:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00100000:3.0:1789302966.546634:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350612992:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302966.546659:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350612992:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302966.546665:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00100000:3.0:1789302966.546677:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 619 00000100:00100000:3.0:1789302966.546838:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350613504:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302966.546869:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350613504:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 185us (683us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302966.546876:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 619 00000100:00100000:3.0:1789302966.546888:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 620 00000100:00100000:3.0:1789302966.546894:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350613120:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302966.546927:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350613120:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (708us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302966.546932:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 620 00000100:00100000:3.0:1789302966.547003:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 621 00000100:00100000:3.0:1789302966.547009:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350613376:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302966.547034:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350613376:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (792us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302966.547040:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 621 00000100:00100000:0.0:1789302966.547186:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350612736 00000100:00100000:0.0:1789302966.547192:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350612736 00000100:00100000:0.0:1789302966.547233:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 652 00000100:00100000:0.0:1789302966.547240:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350612736:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302966.547266:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350612736:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (1178us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302966.547272:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 652 00000100:00080000:0.0:1789302966.547440:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302966.547445:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302966.547447:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789302966.924854:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302967.252545:0:24654:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302967.252554:0:24654:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302967.254170:0:24654:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302967.254186:0:24654:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302967.949600:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302968.972735:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302969.094679:0:24705:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302969.094692:0:24705:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302969.096998:0:24705:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302969.097016:0:24705:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302969.998232:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302970.389205:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371208192:0@lo:400:kworker.0 00000100:00100000:1.0:1789302970.389401:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371208064:0@lo:400:kworker.0 00000100:00100000:1.0:1789302970.389456:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302970.389535:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371208576:0@lo:400:kworker.0 00000100:00100000:1.0:1789302970.389565:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302970.389609:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208576 00000100:00100000:1.0:1789302970.389616:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208576 00000100:00100000:1.0:1789302970.389670:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 584 00000100:00100000:1.0:1789302970.389679:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371208576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302970.389732:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371208576:12345-0@lo:400:kworker.0 Request processed in 54us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302970.389740:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 584 00000100:00080000:1.0:1789302970.389760:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302970.389788:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371208576:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.390037:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302970.390060:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f095ae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371208704:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.390086:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302970.390114:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371208320:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.390137:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302970.390153:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302970.390159:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371208448:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.390178:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302970.390204:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208192 00000100:00100000:3.0:1789302970.390208:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208192 00000100:00100000:0.0:1789302970.390344:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208064 00000100:00100000:0.0:1789302970.390348:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208064 00000100:00100000:0.0:1789302970.390375:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:0.0:1789302970.390382:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371208064:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302970.390422:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371208064:12345-0@lo:400:kworker.0 Request processed in 41us (966us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302970.390431:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00100000:3.0:1789302970.394167:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208320 00000100:00100000:3.0:1789302970.394186:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208320 00000100:00100000:3.0:1789302970.394299:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 585 00000100:00100000:3.0:1789302970.394310:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371208192:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.394404:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371208192:12345-0@lo:400:kworker.0 Request processed in 95us (4366us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302970.394411:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 585 00000100:00100000:3.0:1789302970.394428:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1789302970.394434:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371208320:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.394471:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371208320:12345-0@lo:400:kworker.0 Request processed in 37us (4332us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302970.394477:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1789302970.394524:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208704 00000100:00100000:3.0:1789302970.394529:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208704 00000100:00100000:3.0:1789302970.394559:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208448 00000100:00100000:3.0:1789302970.394562:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208448 00000100:00100000:3.0:1789302970.394582:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:3.0:1789302970.394588:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371208704:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.394624:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371208704:12345-0@lo:400:kworker.0 Request processed in 36us (4537us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302970.394630:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:3.0:1789302970.394642:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:3.0:1789302970.394648:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371208448:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.394677:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371208448:12345-0@lo:400:kworker.0 Request processed in 29us (4498us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302970.394683:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00000100:00100000:3.0:1789302970.394734:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371208192:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.394748:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f095ae300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371208704:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.394785:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371208448:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.394795:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371208320:0@lo:400:kworker.0 00000100:00100000:3.0:1789302970.395011:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371208064:0@lo:400:kworker.0 00000100:00080000:0.0:1789302970.395416:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789302970.395422:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302970.395424:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:1.0:1789302970.573112:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302970.573123:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302970.576775:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302970.576793:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789302970.893548:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302970.893570:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371208832:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789302970.893634:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302970.895986:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208832 00000100:00100000:0.0:1789302970.896000:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208832 00000100:00100000:0.0:1789302970.896058:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:0.0:1789302970.896068:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371208832:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302970.896108:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302970.896189:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371208832:12345-0@lo:13:osp-pre-1-0.0 Request processed in 123us (2553us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302970.896196:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1789302970.896333:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371208832:0@lo:13:osp-pre-1-0.0 00000040:00080000:3.0:1789302970.968690:0:24756:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302970.968698:0:24756:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302970.969258:0:24756:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302970.969270:0:24756:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302971.024235:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302971.405761:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371208960:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302971.405829:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302971.406268:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371208960 00000100:00100000:0.0:1789302971.406276:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371208960 00000100:00100000:0.0:1789302971.406322:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 251 00000100:00100000:0.0:1789302971.406332:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371208960:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302971.406367:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302971.406421:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f757800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371208960:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (591us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302971.406428:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 251 00000100:00100000:0.0:1789302971.406471:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371208960:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302971.664379:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302971.664430:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302971.664473:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302971.664533:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302971.664541:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614144 00000100:00100000:2.0:1789302971.664551:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614144 00000100:00100000:0.0:1789302971.664557:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302971.664576:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302971.664618:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789302971.664620:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350613760 00000100:00100000:2.0:1789302971.664625:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350613760 00000100:00100000:0.0:1789302971.664646:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350613888 00000100:00100000:0.0:1789302971.664652:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350613888 00000100:00100000:2.0:1789302971.664656:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 587 00000100:00100000:2.0:1789302971.664668:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350614144:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302971.664684:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614400 00000100:00100000:0.0:1789302971.664688:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614400 00000100:00100000:0.0:1789302971.664702:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614016 00000100:00100000:0.0:1789302971.664704:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614016 00000100:00100000:0.0:1789302971.664718:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614272 00000100:00100000:0.0:1789302971.664720:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614272 00000100:00100000:0.0:1789302971.664734:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 624 00000100:00100000:0.0:1789302971.664741:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350613888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302971.664748:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de4e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350614144:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 79us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302971.664775:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350613888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (344us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302971.664783:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 624 00000100:00100000:0.0:1789302971.664798:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 625 00000100:00100000:0.0:1789302971.664804:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350614400:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302971.664829:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350614400:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (322us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302971.664835:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 587 00000100:00100000:0.0:1789302971.664841:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 625 00000100:00100000:2.0:1789302971.664856:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00100000:2.0:1789302971.664875:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350613760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302971.664903:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f025500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350613760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 40us (347us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302971.664908:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 588 00000100:00080000:2.0:1789302971.664925:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302971.664928:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302971.664931:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302971.665131:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 626 00000100:00100000:1.0:1789302971.665141:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350614016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302971.665171:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350614016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (593us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302971.665179:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 627 00000100:00100000:3.0:1789302971.665234:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350613632 00000100:00100000:3.0:1789302971.665239:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350613632 00000100:00100000:3.0:1789302971.665276:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 654 00000100:00100000:3.0:1789302971.665283:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350613632:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302971.665311:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350613632:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (1379us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302971.665317:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 654 00000100:00100000:0.0:1789302971.665754:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350614272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302971.666715:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350614272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1522us (2093us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302971.669198:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 626 00000100:00100000:0.0:1789302971.669393:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 627 00010000:00010000:3.0:1789302972.045222:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302972.905358:0:24808:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302972.905367:0:24808:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302972.908241:0:24808:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302972.908260:0:24808:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302973.072164:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302974.092729:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302974.884820:0:24859:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302974.886888:0:24859:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302974.887435:0:24859:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302974.887449:0:24859:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302975.117383:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302975.501617:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371209472:0@lo:400:kworker.0 00000100:00100000:0.0:1789302975.501691:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302975.501747:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789302975.501754:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371209344:0@lo:400:kworker.0 00000100:00100000:0.0:1789302975.501780:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302975.501815:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209472 00000100:00100000:0.0:1789302975.501821:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209472 00000100:00100000:0.0:1789302975.501881:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 628 00000100:00100000:0.0:1789302975.501890:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371209472:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302975.501941:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371209472:12345-0@lo:400:kworker.0 Request processed in 53us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302975.501948:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 628 00000100:00100000:0.0:1789302975.501971:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209344 00000100:00100000:0.0:1789302975.501975:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209344 00000100:00100000:0.0:1789302975.502003:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 589 00000100:00100000:0.0:1789302975.502009:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371209344:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302975.502037:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371209344:12345-0@lo:400:kworker.0 Request processed in 28us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302975.502043:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 589 00000100:00080000:0.0:1789302975.502058:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302975.502062:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302975.502092:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0290b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371209344:0@lo:400:kworker.0 00000100:00100000:0.0:1789302975.502102:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7d180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371209472:0@lo:400:kworker.0 00000100:00100000:2.0:1789302975.502647:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371209216:0@lo:400:kworker.0 00000100:00100000:2.0:1789302975.502689:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302975.502717:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371209728:0@lo:400:kworker.0 00000100:00100000:2.0:1789302975.502740:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302975.502757:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1789302975.502775:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371209088:0@lo:400:kworker.0 00000100:00100000:2.0:1789302975.502801:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302975.502819:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef7c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371209600:0@lo:400:kworker.0 00000100:00100000:2.0:1789302975.502848:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302975.503555:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209216 00000100:00100000:3.0:1789302975.503563:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209216 00000100:00100000:3.0:1789302975.503604:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209600 00000100:00100000:3.0:1789302975.503608:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209600 00000100:00100000:3.0:1789302975.503644:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 590 00000100:00100000:3.0:1789302975.503652:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371209216:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789302975.503810:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302975.503847:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209728 00000100:00100000:1.0:1789302975.503851:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209728 00000100:00100000:1.0:1789302975.503888:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:1.0:1789302975.503895:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371209728:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302975.503947:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371209728:12345-0@lo:400:kworker.0 Request processed in 53us (1205us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302975.503954:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00080000:1.0:1789302975.503975:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789302975.503999:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371209728:0@lo:400:kworker.0 00000100:00100000:1.0:1789302975.504031:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 591 00000100:00100000:1.0:1789302975.504039:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371209600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302975.504077:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371209600:12345-0@lo:400:kworker.0 Request processed in 39us (1228us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302975.504082:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 591 00000100:00100000:1.0:1789302975.504106:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef7c000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371209600:0@lo:400:kworker.0 00000100:00100000:3.0:1789302975.504181:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371209216:12345-0@lo:400:kworker.0 Request processed in 530us (1490us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302975.504187:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 590 00000100:00100000:1.0:1789302975.504357:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371209216:0@lo:400:kworker.0 00000100:00100000:0.0:1789302975.504478:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209088 00000100:00100000:0.0:1789302975.504483:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209088 00000100:00100000:0.0:1789302975.504541:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:0.0:1789302975.504548:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371209088:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302975.504585:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371209088:12345-0@lo:400:kworker.0 Request processed in 38us (1783us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302975.504591:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:1.0:1789302975.505039:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371209088:0@lo:400:kworker.0 00000100:00100000:2.0:1789302976.012825:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371209856:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302976.012902:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302976.013215:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209856 00000100:00100000:0.0:1789302976.013223:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209856 00000100:00100000:0.0:1789302976.013277:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:0.0:1789302976.013287:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371209856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302976.013320:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302976.013386:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371209856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 100us (483us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302976.013394:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:2.0:1789302976.014603:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371209856:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789302976.141684:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302976.524824:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302976.524856:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371209984:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302976.524919:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302976.525034:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371209984 00000100:00100000:0.0:1789302976.525043:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371209984 00000100:00100000:0.0:1789302976.525105:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 253 00000100:00100000:0.0:1789302976.525114:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371209984:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302976.525148:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302976.525201:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371209984:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (281us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302976.525207:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 253 00000100:00100000:3.0:1789302976.525280:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371209984:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302976.784635:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302976.784778:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302976.785226:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302976.785253:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302976.785271:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302976.785290:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302976.785384:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302976.785445:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614656 00000100:00100000:3.0:1789302976.785454:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614656 00000100:00100000:0.0:1789302976.788122:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614528 00000100:00100000:0.0:1789302976.788137:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614528 00000100:00100000:0.0:1789302976.788207:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 656 00000100:00100000:0.0:1789302976.788218:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350614528:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302976.788299:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350614528:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 82us (3044us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302976.788306:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 656 00000100:00100000:0.0:1789302976.788341:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614912 00000100:00100000:0.0:1789302976.788345:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614912 00000100:00100000:0.0:1789302976.793077:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615168 00000100:00100000:0.0:1789302976.793090:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615168 00000100:00080000:2.0:1789302976.794064:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789302976.794071:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302976.794075:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302976.794136:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350614784 00000100:00100000:1.0:1789302976.794143:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350614784 00000100:00100000:1.0:1789302976.794201:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615296 00000100:00100000:1.0:1789302976.794205:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615296 00000100:00100000:1.0:1789302976.794227:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 630 00000100:00100000:1.0:1789302976.794237:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350614912:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302976.794312:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127b480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350614912:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 77us (9534us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302976.794320:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 630 00000100:00100000:1.0:1789302976.794339:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 631 00000100:00100000:1.0:1789302976.794346:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350615168:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302976.794370:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350615168:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (9145us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302976.794377:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 631 00000100:00100000:1.0:1789302976.794389:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 632 00000100:00100000:1.0:1789302976.794395:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350614784:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302976.794418:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350614784:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (9145us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302976.794424:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 632 00000100:00100000:1.0:1789302976.794436:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 633 00000100:00100000:1.0:1789302976.794443:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350615296:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302976.794466:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350615296:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (9152us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302976.794476:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 633 00000100:00100000:3.0:1789302976.795097:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615040 00000100:00100000:3.0:1789302976.795105:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615040 00000100:00100000:3.0:1789302976.795131:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 592 00000100:00100000:3.0:1789302976.795142:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350614656:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302976.795184:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f095ae300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350614656:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 46us (10560us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302976.795191:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 592 00000100:00100000:3.0:1789302976.796185:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 593 00000100:00100000:3.0:1789302976.796196:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350615040:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302976.796242:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350615040:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 48us (10950us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302976.796249:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 593 00000100:00080000:2.0:1789302976.800756:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789302976.800766:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789302976.800770:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:3.0:1789302976.819127:0:24910:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302976.819435:0:24910:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302976.820746:0:24910:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302976.820766:0:24910:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302977.164986:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302978.189119:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302978.867898:0:24962:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302978.867908:0:24962:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302978.868438:0:24962:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302978.868449:0:24962:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789302979.212651:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789302980.238249:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302980.622644:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371210240:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.622714:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302980.622762:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371210752:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.622789:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302980.622825:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789302980.622831:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371210112:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.622859:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302980.622882:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371210624:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.622904:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302980.622937:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371210368:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.622961:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302980.622979:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302980.622987:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371210496:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623007:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302980.623036:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371210752 00000100:00100000:1.0:1789302980.623043:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371210752 00000100:00100000:1.0:1789302980.623161:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371210496 00000100:00100000:1.0:1789302980.623166:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371210496 00000100:00100000:1.0:1789302980.623195:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:1.0:1789302980.623204:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371210752:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623256:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371210752:12345-0@lo:400:kworker.0 Request processed in 53us (466us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302980.623263:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:1.0:1789302980.623281:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:1.0:1789302980.623287:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371210496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623322:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371210496:12345-0@lo:400:kworker.0 Request processed in 35us (313us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302980.623328:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:1.0:1789302980.623346:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371210112 00000100:00100000:1.0:1789302980.623350:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371210112 00000100:00100000:1.0:1789302980.623371:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:1.0:1789302980.623377:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371210112:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623407:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f6050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371210112:12345-0@lo:400:kworker.0 Request processed in 31us (547us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302980.623413:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:1.0:1789302980.623440:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371210240 00000100:00100000:1.0:1789302980.623444:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371210240 00000100:00100000:1.0:1789302980.623467:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371210624 00000100:00100000:1.0:1789302980.623470:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371210624 00000100:00100000:1.0:1789302980.623487:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371210368 00000100:00100000:1.0:1789302980.623526:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371210368 00000100:00100000:1.0:1789302980.623544:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 594 00000100:00100000:1.0:1789302980.623550:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371210240:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623580:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371210240:12345-0@lo:400:kworker.0 Request processed in 30us (867us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302980.623586:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 594 00000100:00100000:1.0:1789302980.623598:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 595 00000100:00100000:1.0:1789302980.623604:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371210624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623652:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371210624:12345-0@lo:400:kworker.0 Request processed in 48us (747us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302980.623662:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 595 00000100:00100000:1.0:1789302980.623675:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 596 00000100:00100000:1.0:1789302980.623681:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371210368:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623708:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371210368:12345-0@lo:400:kworker.0 Request processed in 27us (746us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302980.623714:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 596 00000100:00100000:1.0:1789302980.623758:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371210496:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623769:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371210368:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623796:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371210112:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623819:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371210624:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623828:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371210240:0@lo:400:kworker.0 00000100:00100000:1.0:1789302980.623836:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6adf80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371210752:0@lo:400:kworker.0 00000100:00080000:3.0:1789302980.624914:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789302980.624923:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302980.624925:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000040:00080000:3.0:1789302980.824962:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302980.824972:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302980.825663:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302980.825674:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:1.0:1789302980.929832:0:25013:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302980.929842:0:25013:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302980.930668:0:25013:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302980.930680:0:25013:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302981.132812:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302981.132837:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371210880:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302981.132918:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302981.133068:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371210880 00000100:00100000:0.0:1789302981.133077:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371210880 00000100:00100000:0.0:1789302981.133125:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:0.0:1789302981.133134:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371210880:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302981.133167:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302981.133211:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371210880:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (293us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302981.133218:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:2.0:1789302981.133695:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371210880:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302981.260875:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302981.646291:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302981.646317:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371211008:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789302981.646385:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302981.647174:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211008 00000100:00100000:0.0:1789302981.647185:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211008 00000100:00100000:0.0:1789302981.647244:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 255 00000100:00100000:0.0:1789302981.647254:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371211008:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302981.647290:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302981.647361:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371211008:12345-0@lo:13:osp-pre-0-0.0 Request processed in 108us (975us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302981.647369:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 255 00000100:00100000:2.0:1789302981.647549:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371211008:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789302981.900600:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302981.900640:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302981.900806:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615552 00000100:00100000:1.0:1789302981.900815:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615552 00000100:00080000:3.0:1789302981.900961:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302981.901392:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 597 00000100:00100000:1.0:1789302981.901406:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350615552:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302981.901467:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615424 00000100:00100000:1.0:1789302981.901481:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127aa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350615552:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 79us (839us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302981.901485:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615424 00000100:00100000:1.0:1789302981.901491:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 597 00000100:00100000:0.0:1789302981.901545:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 658 00000100:00100000:0.0:1789302981.901553:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350615424:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302981.901586:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350615424:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (994us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302981.901594:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 658 00000100:00100000:3.0:1789302981.903619:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302981.903661:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302981.903704:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302981.903741:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302981.903744:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615808 00000100:00100000:0.0:1789302981.903755:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615808 00000100:00100000:3.0:1789302981.903768:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615936 00000100:00100000:3.0:1789302981.903774:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615936 00000100:00100000:0.0:1789302981.903830:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616064 00000100:00100000:0.0:1789302981.903834:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616064 00000100:00100000:2.0:1789302981.908355:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350615680 00000100:00100000:2.0:1789302981.908370:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350615680 00000100:00100000:1.0:1789302981.908471:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 636 00000100:00100000:1.0:1789302981.908484:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350615808:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302981.908595:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350615808:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 114us (4984us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302981.908604:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 636 00000100:00100000:3.0:1789302981.908933:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789302981.909131:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616192 00000100:00100000:3.0:1789302981.909136:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616192 00000100:00100000:3.0:1789302981.909177:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 637 00000100:00100000:3.0:1789302981.909185:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01278380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350616192:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302981.909222:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01278380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350616192:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 38us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302981.909228:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 637 00000100:00100000:3.0:1789302981.909323:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 598 00000100:00100000:3.0:1789302981.909332:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01278a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350615936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302981.909364:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01278a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350615936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (5624us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302981.909370:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 598 00000100:00100000:0.0:1789302981.911399:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 638 00000100:00100000:0.0:1789302981.911413:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350616064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302981.911464:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 639 00000100:00100000:2.0:1789302981.911475:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350615680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:1.0:1789302981.911594:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302981.911600:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302981.911602:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789302981.913245:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350616064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1825us (9571us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302981.913251:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350615680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 1775us (9549us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302981.913261:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 638 00000100:00100000:2.0:1789302981.913269:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 639 00010000:00010000:0.0:1789302982.294544:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302982.865743:0:25064:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302982.865754:0:25064:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302982.866299:0:25064:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302982.866308:0:25064:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302983.308727:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302984.334119:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302985.003568:0:25115:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302985.003578:0:25115:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302985.006393:0:25115:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302985.006418:0:25115:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789302985.356808:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302985.742240:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371211136:0@lo:400:kworker.0 00000100:00100000:1.0:1789302985.742314:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302985.742470:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f026d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371211520:0@lo:400:kworker.0 00000100:00100000:0.0:1789302985.742526:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302985.742552:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371211392:0@lo:400:kworker.0 00000100:00100000:0.0:1789302985.742579:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302985.742598:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f41c90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371211264:0@lo:400:kworker.0 00000100:00100000:0.0:1789302985.742618:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302985.742640:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211392 00000100:00100000:0.0:1789302985.742648:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211392 00000100:00100000:0.0:1789302985.742710:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211264 00000100:00100000:0.0:1789302985.742713:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211264 00000100:00100000:0.0:1789302985.742733:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 599 00000100:00100000:0.0:1789302985.742751:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371211392:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302985.743019:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371211392:12345-0@lo:400:kworker.0 Request processed in 269us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302985.743028:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 599 00000100:00100000:0.0:1789302985.743046:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 600 00000100:00100000:0.0:1789302985.743053:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371211264:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302985.743084:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371211264:12345-0@lo:400:kworker.0 Request processed in 32us (465us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302985.743090:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 600 00000100:00080000:0.0:1789302985.743118:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302985.743130:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211136 00000100:00100000:0.0:1789302985.743133:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211136 00000100:00100000:0.0:1789302985.743155:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:0.0:1789302985.743160:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371211136:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302985.743189:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f5850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371211136:12345-0@lo:400:kworker.0 Request processed in 29us (876us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302985.743195:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:3.0:1789302985.744103:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211520 00000100:00100000:3.0:1789302985.744112:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211520 00000100:00100000:3.0:1789302985.744207:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:3.0:1789302985.744216:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371211520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302985.744283:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371211520:12345-0@lo:400:kworker.0 Request processed in 69us (1757us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302985.744289:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00080000:3.0:1789302985.744315:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302985.745367:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c92300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371211136:0@lo:400:kworker.0 00000100:00100000:1.0:1789302985.745381:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f024e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371211648:0@lo:400:kworker.0 00000100:00100000:1.0:1789302985.745426:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302985.746018:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211648 00000100:00100000:0.0:1789302985.746026:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211648 00000100:00100000:0.0:1789302985.746088:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 601 00000100:00100000:0.0:1789302985.746098:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371211648:12345-0@lo:400:kworker.0 00000100:00080000:2.0:1789302985.746157:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302985.746420:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371211648:12345-0@lo:400:kworker.0 Request processed in 325us (993us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302985.746428:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 601 00000100:00100000:1.0:1789302985.749088:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ad44700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371211392:0@lo:400:kworker.0 00000100:00100000:2.0:1789302985.749148:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f41c90000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371211264:0@lo:400:kworker.0 00000100:00100000:3.0:1789302985.749156:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f026d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371211520:0@lo:400:kworker.0 00000100:00100000:2.0:1789302985.749162:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371211776:0@lo:400:kworker.0 00000100:00100000:3.0:1789302985.749211:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f024e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371211648:0@lo:400:kworker.0 00000100:00100000:2.0:1789302985.749229:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302985.749283:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211776 00000100:00100000:1.0:1789302985.749291:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211776 00000100:00100000:1.0:1789302985.749357:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:1.0:1789302985.749366:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371211776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789302985.749424:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371211776:12345-0@lo:400:kworker.0 Request processed in 59us (195us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302985.749432:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00080000:2.0:1789302985.749499:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302985.749561:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f027b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371211776:0@lo:400:kworker.0 00000100:00100000:2.0:1789302986.252797:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302986.252827:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371211904:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302986.252938:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302986.253151:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371211904 00000100:00100000:0.0:1789302986.253163:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371211904 00000100:00100000:0.0:1789302986.253235:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:0.0:1789302986.253247:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371211904:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789302986.253303:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789302986.253365:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371211904:12345-0@lo:13:osp-pre-1-0.0 Request processed in 121us (427us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302986.253372:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:0.0:1789302986.253450:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371211904:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789302986.380684:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302986.765530:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371212032:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302986.765623:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302986.767054:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212032 00000100:00100000:3.0:1789302986.767073:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212032 00000100:00100000:3.0:1789302986.767952:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 257 00000100:00100000:3.0:1789302986.767969:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371212032:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302986.768026:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302986.768582:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef4700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371212032:12345-0@lo:13:osp-pre-0-0.0 Request processed in 617us (2956us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302986.768591:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 257 00000100:00100000:1.0:1789302986.768702:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371212032:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302987.023895:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302987.023953:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302987.023988:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302987.024014:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302987.024040:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302987.024068:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302987.024163:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302987.024202:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616576 00000100:00100000:0.0:1789302987.024216:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616576 00000100:00100000:0.0:1789302987.024290:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617088 00000100:00100000:0.0:1789302987.024293:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617088 00000100:00100000:0.0:1789302987.024318:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616704 00000100:00100000:0.0:1789302987.024321:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616704 00000100:00100000:0.0:1789302987.024338:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616960 00000100:00100000:0.0:1789302987.024342:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616960 00000100:00100000:0.0:1789302987.024362:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00100000:0.0:1789302987.024375:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350616576:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302987.024475:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350616576:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 101us (520us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302987.024484:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00100000:0.0:1789302987.024533:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 643 00000100:00100000:0.0:1789302987.024541:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350617088:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302987.024570:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350617088:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (555us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302987.024577:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 643 00000100:00100000:0.0:1789302987.024589:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 644 00000100:00100000:0.0:1789302987.024596:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350616704:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302987.024621:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25c44a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350616704:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302987.024627:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 644 00000100:00100000:0.0:1789302987.024640:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 645 00000100:00100000:0.0:1789302987.024646:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350616960:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302987.024669:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350616960:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (506us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302987.024675:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 645 00000100:00100000:0.0:1789302987.024697:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616832 00000100:00100000:0.0:1789302987.024701:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616832 00000100:00100000:0.0:1789302987.024732:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616448 00000100:00100000:0.0:1789302987.024735:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616448 00000100:00100000:0.0:1789302987.024769:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 602 00000100:00100000:0.0:1789302987.024777:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350616832:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302987.024808:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef7c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350616832:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (819us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302987.024814:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 602 00000100:00100000:0.0:1789302987.024826:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 603 00000100:00100000:0.0:1789302987.024833:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350616448:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302987.024857:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0290b100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350616448:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (817us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302987.024863:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 603 00000100:00080000:0.0:1789302987.024896:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789302987.024901:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302987.024904:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302987.026454:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350616320 00000100:00100000:1.0:1789302987.026471:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350616320 00000100:00100000:1.0:1789302987.026557:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 660 00000100:00100000:1.0:1789302987.026570:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350616320:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302987.026673:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350616320:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 107us (2794us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302987.026681:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 660 00000040:00080000:1.0:1789302987.099746:0:25167:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302987.099773:0:25167:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302987.107109:0:25167:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302987.107137:0:25167:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302987.414743:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1789302988.428771:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302989.151005:0:25218:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302989.151017:0:25218:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302989.151560:0:25218:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302989.151572:0:25218:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302989.453555:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302990.028916:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302990.028930:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302990.029434:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302990.029463:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302990.479608:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789302990.862992:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01278e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371212416:0@lo:400:kworker.0 00000100:00100000:0.0:1789302990.863029:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371212160:0@lo:400:kworker.0 00000100:00100000:3.0:1789302990.863039:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371212288:0@lo:400:kworker.0 00000100:00100000:0.0:1789302990.863355:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302990.863356:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302990.863428:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371212800:0@lo:400:kworker.0 00000100:00100000:0.0:1789302990.863430:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01279c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371212672:0@lo:400:kworker.0 00000100:00100000:3.0:1789302990.863471:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302990.863478:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302990.863553:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212160 00000100:00100000:0.0:1789302990.863563:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212160 00000100:00100000:0.0:1789302990.863618:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:0.0:1789302990.863628:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371212160:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302990.864472:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212288 00000100:00100000:3.0:1789302990.864483:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212288 00000100:00100000:3.0:1789302990.865031:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212672 00000100:00100000:3.0:1789302990.865037:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212672 00000100:00100000:3.0:1789302990.865082:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 604 00000100:00100000:0.0:1789302990.865082:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371212160:12345-0@lo:400:kworker.0 Request processed in 1450us (1728us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302990.865093:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01278700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371212288:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302990.865098:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00100000:0.0:1789302990.865155:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:3.0:1789302990.865157:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01278700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371212288:12345-0@lo:400:kworker.0 Request processed in 63us (1802us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302990.865168:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 604 00000100:00100000:0.0:1789302990.865169:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371212672:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302990.865197:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212800 00000100:00100000:3.0:1789302990.865202:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212800 00000100:00100000:0.0:1789302990.865222:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371212672:12345-0@lo:400:kworker.0 Request processed in 57us (1744us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302990.865231:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00080000:0.0:1789302990.865252:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789302990.865258:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789302990.865292:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371212160:0@lo:400:kworker.0 00000100:00100000:0.0:1789302990.865309:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01279c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371212672:0@lo:400:kworker.0 00000100:00100000:3.0:1789302990.865671:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:3.0:1789302990.865682:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371212800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302990.865746:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371212800:12345-0@lo:400:kworker.0 Request processed in 67us (2275us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302990.865753:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:1.0:1789302990.865817:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302990.865865:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789302990.865880:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01278000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371212544:0@lo:400:kworker.0 00000100:00100000:1.0:1789302990.865917:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789302990.869639:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212416 00000100:00100000:2.0:1789302990.869672:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212416 00000100:00100000:2.0:1789302990.869776:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:2.0:1789302990.869789:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371212416:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302990.869908:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371212416:12345-0@lo:400:kworker.0 Request processed in 122us (4090us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302990.869916:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:2.0:1789302990.869987:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371212288:0@lo:400:kworker.0 00000100:00100000:2.0:1789302990.870004:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371212800:0@lo:400:kworker.0 00000100:00080000:1.0:1789302990.871090:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789302990.871163:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01278e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371212416:0@lo:400:kworker.0 00000100:00100000:0.0:1789302990.872050:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212544 00000100:00100000:0.0:1789302990.872063:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212544 00000100:00100000:0.0:1789302990.872152:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:0.0:1789302990.872163:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371212544:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302990.872278:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371212544:12345-0@lo:400:kworker.0 Request processed in 117us (6359us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302990.872288:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00080000:1.0:1789302990.872483:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302990.872551:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01278000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371212544:0@lo:400:kworker.0 00000040:00080000:3.0:1789302991.046663:0:25269:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302991.046676:0:25269:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302991.047995:0:25269:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302991.048014:0:25269:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789302991.373231:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789302991.373265:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371212928:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789302991.373381:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789302991.373450:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371212928 00000100:00100000:1.0:1789302991.373461:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371212928 00000100:00100000:2.0:1789302991.376311:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:2.0:1789302991.376347:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371212928:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789302991.376410:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789302991.376592:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef6a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371212928:12345-0@lo:13:osp-pre-1-0.0 Request processed in 259us (3204us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302991.376600:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00100000:1.0:1789302991.376968:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371212928:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789302991.501063:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789302991.885264:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371213056:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789302991.885362:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302991.886141:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213056 00000100:00100000:3.0:1789302991.886154:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213056 00000100:00100000:3.0:1789302991.886225:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 259 00000100:00100000:3.0:1789302991.886238:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371213056:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789302991.886298:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789302991.886387:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03ef7b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371213056:12345-0@lo:13:osp-pre-0-0.0 Request processed in 152us (1025us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302991.886394:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 259 00000100:00100000:1.0:1789302991.890426:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371213056:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302992.142359:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302992.142572:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302992.142644:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302992.144173:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302992.144233:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302992.144267:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302992.144313:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789302992.144368:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617472 00000100:00100000:0.0:1789302992.144382:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617472 00000100:00100000:0.0:1789302992.144468:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617984 00000100:00100000:0.0:1789302992.144473:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617984 00000100:00100000:0.0:1789302992.144800:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617600 00000100:00100000:0.0:1789302992.144805:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617600 00000100:00100000:0.0:1789302992.144834:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617856 00000100:00100000:0.0:1789302992.144837:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617856 00000100:00100000:0.0:1789302992.144856:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:0.0:1789302992.144867:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350617472:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302992.144868:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 649 00000100:00100000:1.0:1789302992.144877:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350617984:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302992.144969:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617728 00000100:00100000:3.0:1789302992.144974:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617728 00000100:00100000:3.0:1789302992.145009:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617344 00000100:00100000:3.0:1789302992.145012:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617344 00000100:00100000:3.0:1789302992.145033:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 607 00000100:00100000:3.0:1789302992.145040:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350617728:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789302992.145110:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350617472:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 244us (2538us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302992.145119:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:0.0:1789302992.145139:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 650 00000100:00100000:0.0:1789302992.145146:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350617600:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302992.147952:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350617984:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 3057us (3773us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302992.147971:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 649 00000100:00100000:1.0:1789302992.148020:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 651 00000100:00100000:1.0:1789302992.148035:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350617856:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302992.148085:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70f100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350617856:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 58us (3773us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302992.148090:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 651 00000100:00080000:1.0:1789302992.148118:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302992.148125:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789302992.148128:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789302992.148271:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:1.0:1789302992.148281:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350617344:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302992.148311:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350617344:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 32us (4079us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302992.148315:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 608 00000100:00100000:2.0:1789302992.150659:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350617216 00000100:00100000:2.0:1789302992.150680:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350617216 00000100:00100000:3.0:1789302992.151127:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350617728:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 6077us (8472us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302992.151143:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 607 00000100:00100000:2.0:1789302992.152081:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00100000:2.0:1789302992.152099:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350617216:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789302992.152409:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f7850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350617216:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 314us (10065us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302992.152418:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00100000:0.0:1789302992.155931:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02b09500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350617600:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 10764us (11643us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302992.155952:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 650 00010000:00010000:3.0:1789302992.524877:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789302993.104739:0:25321:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789302993.104754:0:25321:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789302993.105291:0:25321:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789302993.105306:0:25321:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302993.549038:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789302994.575253:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789302995.111655:0:25372:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789302995.111668:0:25372:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789302995.113398:0:25372:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789302995.113422:0:25372:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789302995.596819:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789302995.983352:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371213440:0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.983449:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302995.983555:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789302995.983562:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371213568:0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.983592:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302995.983604:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01278000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371213184:0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.983641:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213440 00000100:00100000:3.0:1789302995.983653:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213440 00000100:00100000:0.0:1789302995.983659:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302995.983691:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01279c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371213696:0@lo:400:kworker.0 00000100:00100000:0.0:1789302995.983730:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302995.983750:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789302995.983757:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01278e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371213312:0@lo:400:kworker.0 00000100:00100000:0.0:1789302995.983782:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789302995.983862:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213696 00000100:00100000:3.0:1789302995.983868:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213696 00000100:00100000:3.0:1789302995.983902:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:3.0:1789302995.983912:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371213440:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.983990:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127b480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371213440:12345-0@lo:400:kworker.0 Request processed in 78us (541us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302995.983998:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:3.0:1789302995.984018:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:3.0:1789302995.984025:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371213696:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.984053:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70fb80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371213696:12345-0@lo:400:kworker.0 Request processed in 29us (324us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302995.984060:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:3.0:1789302995.984080:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213568 00000100:00100000:3.0:1789302995.984084:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213568 00000100:00100000:3.0:1789302995.984116:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:3.0:1789302995.984123:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371213568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.984156:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127bb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371213568:12345-0@lo:400:kworker.0 Request processed in 34us (564us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302995.984163:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00080000:3.0:1789302995.984180:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789302995.984185:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789302995.984218:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127b100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371213568:0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.984229:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127a300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371213440:0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.984587:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213184 00000100:00100000:3.0:1789302995.984592:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213184 00000100:00100000:3.0:1789302995.984621:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 663 00000100:00100000:3.0:1789302995.984628:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371213184:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789302995.984667:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f4c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371213184:12345-0@lo:400:kworker.0 Request processed in 40us (1009us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302995.984673:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 663 00000100:00100000:0.0:1789302995.988832:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371213824:0@lo:400:kworker.0 00000100:00100000:0.0:1789302995.988914:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302995.988970:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01278000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371213184:0@lo:400:kworker.0 00000100:00100000:0.0:1789302995.988991:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01279c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371213696:0@lo:400:kworker.0 00000100:00100000:2.0:1789302995.989060:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213312 00000100:00100000:0.0:1789302995.989060:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213824 00000100:00100000:2.0:1789302995.989068:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213312 00000100:00100000:0.0:1789302995.989068:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213824 00000100:00100000:2.0:1789302995.989124:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 611 00000100:00100000:0.0:1789302995.989124:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:0.0:1789302995.989133:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371213824:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789302995.989134:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371213312:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789302995.989193:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371213824:12345-0@lo:400:kworker.0 Request processed in 59us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302995.989201:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00080000:0.0:1789302995.989225:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789302995.994012:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db44380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371213312:12345-0@lo:400:kworker.0 Request processed in 4864us (10214us total) trans 0 rc 0/0 00000100:00100000:2.0:1789302995.994032:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 611 00000100:00100000:0.0:1789302995.999460:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01278e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371213312:0@lo:400:kworker.0 00000100:00100000:1.0:1789302996.000043:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371213824:0@lo:400:kworker.0 00000100:00100000:3.0:1789302996.493079:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371213952:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789302996.493182:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302996.493370:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371213952 00000100:00100000:0.0:1789302996.493381:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371213952 00000100:00100000:0.0:1789302996.493450:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:0.0:1789302996.493461:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371213952:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789302996.494346:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789302996.494566:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371213952:12345-0@lo:13:osp-pre-1-0.0 Request processed in 1103us (1381us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302996.494577:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:0.0:1789302996.494714:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371213952:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789302996.621120:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789302997.007443:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371214080:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789302997.007566:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789302997.007662:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214080 00000100:00100000:0.0:1789302997.007672:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214080 00000100:00100000:0.0:1789302997.007757:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 261 00000100:00100000:0.0:1789302997.007768:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371214080:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789302997.007813:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789302997.007864:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db45500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371214080:12345-0@lo:13:osp-pre-0-0.0 Request processed in 97us (298us total) trans 0 rc 0/0 00000100:00100000:0.0:1789302997.007883:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 261 00000100:00080000:0.0:1789302997.007909:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789302997.007950:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371214080:0@lo:13:osp-pre-0-0.0 00000040:00080000:1.0:1789302997.166993:0:25423:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302997.167006:0:25423:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302997.167606:0:25423:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302997.167619:0:25423:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789302997.260211:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302997.260261:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302997.260288:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302997.260325:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302997.260350:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302997.260371:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302997.260397:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789302997.260432:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350618496 00000100:00100000:1.0:1789302997.260444:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350618496 00000100:00100000:1.0:1789302997.260552:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350618752 00000100:00100000:1.0:1789302997.260556:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350618752 00000100:00100000:1.0:1789302997.260580:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350618368 00000100:00100000:1.0:1789302997.260583:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350618368 00000100:00100000:1.0:1789302997.260595:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350618880 00000100:00100000:1.0:1789302997.260598:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350618880 00000100:00100000:1.0:1789302997.260615:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 654 00000100:00100000:1.0:1789302997.260626:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350618496:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302997.260775:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350618240 00000100:00100000:3.0:1789302997.260780:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350618240 00000100:00100000:3.0:1789302997.260815:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350618624 00000100:00100000:3.0:1789302997.260818:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350618624 00000100:00100000:3.0:1789302997.260842:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:3.0:1789302997.260850:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350618240:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302997.260931:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350618240:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 81us (735us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302997.260938:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 612 00000100:00100000:3.0:1789302997.260956:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 613 00000100:00100000:3.0:1789302997.260963:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350618624:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302997.260990:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350618624:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (618us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302997.260997:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 613 00000100:00100000:3.0:1789302997.261024:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 655 00000100:00100000:3.0:1789302997.261032:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350618752:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789302997.261057:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350618752:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (769us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302997.261064:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 655 00000100:00100000:3.0:1789302997.261082:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 656 00000100:00100000:3.0:1789302997.261089:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350618368:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302997.261179:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350618496:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 554us (917us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302997.261186:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 654 00000100:00100000:1.0:1789302997.261200:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 657 00000100:00100000:1.0:1789302997.261206:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350618880:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302997.261230:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350618880:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (833us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302997.261235:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 657 00000100:00100000:1.0:1789302997.261259:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350618112 00000100:00100000:1.0:1789302997.261263:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350618112 00000100:00100000:1.0:1789302997.261288:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 664 00000100:00100000:1.0:1789302997.261294:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350618112:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789302997.261319:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350618112:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (994us total) trans 0 rc 0/0 00000100:00100000:1.0:1789302997.261326:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 664 00000100:00080000:1.0:1789302997.261358:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789302997.261364:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789302997.261366:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789302997.261440:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350618368:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 351us (1089us total) trans 0 rc 0/0 00000100:00100000:3.0:1789302997.261447:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 656 00010000:00010000:1.0:1789302997.645379:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789302998.670127:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789302999.176626:0:25474:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789302999.176638:0:25474:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789302999.177186:0:25474:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789302999.177198:0:25474:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789302999.694073:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303000.268870:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303000.268883:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303000.269445:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303000.269459:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303000.717343:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789303001.101462:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371214592:0@lo:400:kworker.0 00000100:00100000:0.0:1789303001.101591:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303001.101686:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789303001.101693:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371214464:0@lo:400:kworker.0 00000100:00100000:0.0:1789303001.101725:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303001.102016:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214592 00000100:00100000:0.0:1789303001.102029:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214592 00000100:00100000:0.0:1789303001.102110:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:0.0:1789303001.102122:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371214592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303001.102197:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371214592:12345-0@lo:400:kworker.0 Request processed in 77us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303001.102205:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00000100:00080000:0.0:1789303001.102233:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789303001.102277:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371214592:0@lo:400:kworker.0 00000100:00100000:3.0:1789303001.102369:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214464 00000100:00100000:3.0:1789303001.102375:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214464 00000100:00100000:3.0:1789303001.102425:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:3.0:1789303001.102442:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371214464:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303001.102486:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef79f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371214464:12345-0@lo:400:kworker.0 Request processed in 54us (759us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303001.102493:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00100000:2.0:1789303001.102697:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371214336:0@lo:400:kworker.0 00000100:00100000:2.0:1789303001.102752:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303001.102792:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f755c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371214848:0@lo:400:kworker.0 00000100:00100000:2.0:1789303001.102819:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303001.102841:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1789303001.102848:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371214208:0@lo:400:kworker.0 00000100:00100000:2.0:1789303001.102878:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303001.102899:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371214720:0@lo:400:kworker.0 00000100:00100000:2.0:1789303001.102923:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303001.103056:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214336 00000100:00100000:3.0:1789303001.103063:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214336 00000100:00100000:3.0:1789303001.103099:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214720 00000100:00100000:3.0:1789303001.103103:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214720 00000100:00100000:3.0:1789303001.103126:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:3.0:1789303001.103132:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371214336:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303001.103175:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371214336:12345-0@lo:400:kworker.0 Request processed in 43us (423us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303001.103181:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:3.0:1789303001.103192:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:3.0:1789303001.103198:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371214720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303001.103224:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41fbb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371214720:12345-0@lo:400:kworker.0 Request processed in 26us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303001.103230:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00100000:3.0:1789303001.103250:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214208 00000100:00100000:3.0:1789303001.103254:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214208 00000100:00100000:3.0:1789303001.103281:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:3.0:1789303001.103287:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371214208:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303001.103332:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371214208:12345-0@lo:400:kworker.0 Request processed in 45us (454us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303001.103342:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:3.0:1789303001.103392:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371214208:0@lo:400:kworker.0 00000100:00100000:3.0:1789303001.103404:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371214720:0@lo:400:kworker.0 00000100:00100000:3.0:1789303001.103413:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1db46d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371214336:0@lo:400:kworker.0 00000100:00100000:1.0:1789303001.103675:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214848 00000100:00100000:1.0:1789303001.103680:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214848 00000100:00100000:1.0:1789303001.103718:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:1.0:1789303001.103726:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371214848:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303001.103784:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371214848:12345-0@lo:400:kworker.0 Request processed in 60us (964us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303001.103791:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:1.0:1789303001.103826:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f755c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371214848:0@lo:400:kworker.0 00000100:00080000:0.0:1789303001.103957:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789303001.103963:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303001.103992:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371214464:0@lo:400:kworker.0 00000040:00080000:3.0:1789303001.345578:0:25526:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303001.345591:0:25526:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303001.346149:0:25526:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303001.346162:0:25526:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789303001.612764:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789303001.612798:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371214976:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789303001.612892:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303001.613128:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371214976 00000100:00100000:0.0:1789303001.613139:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371214976 00000100:00100000:0.0:1789303001.613204:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:0.0:1789303001.613215:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371214976:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789303001.613262:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789303001.613318:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f755c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371214976:12345-0@lo:13:osp-pre-1-0.0 Request processed in 104us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303001.613325:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:1.0:1789303001.613437:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120c9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371214976:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789303001.752098:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303002.124836:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303002.124867:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371215104:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303002.124978:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303002.125158:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371215104 00000100:00100000:0.0:1789303002.125170:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371215104 00000100:00100000:0.0:1789303002.125237:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 263 00000100:00100000:0.0:1789303002.125250:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371215104:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789303002.125298:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789303002.125366:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371215104:12345-0@lo:13:osp-pre-0-0.0 Request processed in 119us (388us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303002.125373:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 263 00000100:00100000:2.0:1789303002.125579:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371215104:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303002.386282:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303002.388869:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619136 00000100:00100000:3.0:1789303002.388891:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619136 00000100:00100000:3.0:1789303002.389015:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 617 00000100:00100000:3.0:1789303002.389032:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ad44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350619136:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303002.389150:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ad44700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350619136:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 123us (2891us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303002.389158:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 617 00000100:00080000:0.0:1789303002.389252:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303002.389324:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303002.389392:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303002.389455:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303002.389493:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619008 00000100:00100000:3.0:1789303002.389498:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619008 00000100:00100000:3.0:1789303002.389551:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 666 00000100:00100000:3.0:1789303002.389559:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350619008:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303002.389596:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350619008:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (3257us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303002.389607:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 666 00000100:00100000:3.0:1789303002.395870:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619392 00000100:00100000:0.0:1789303002.395873:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619264 00000100:00100000:3.0:1789303002.395892:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619392 00000100:00100000:0.0:1789303002.395892:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619264 00000100:00100000:0.0:1789303002.396222:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 660 00000100:00100000:0.0:1789303002.396236:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350619264:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303002.396332:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa6300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350619264:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 99us (6875us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303002.396340:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 660 00000100:00100000:1.0:1789303002.396582:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303002.396622:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303002.396665:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303002.396691:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619520 00000100:00100000:1.0:1789303002.396696:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619520 00000100:00100000:0.0:1789303002.396746:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619648 00000100:00100000:0.0:1789303002.396751:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619648 00000100:00080000:2.0:1789303002.405787:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789303002.405806:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303002.405810:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789303002.405887:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619776 00000100:00100000:2.0:1789303002.405903:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619776 00000100:00100000:2.0:1789303002.406015:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 661 00000100:00100000:2.0:1789303002.406029:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350619776:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303002.406153:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350619776:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 127us (9486us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303002.406161:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 661 00000100:00080000:2.0:1789303002.406187:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789303002.412356:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00100000:3.0:1789303002.412377:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350619392:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303002.412499:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa5880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350619392:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 128us (23105us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303002.412533:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00100000:1.0:1789303002.413888:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00100000:1.0:1789303002.413903:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350619520:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303002.413975:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c7b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350619520:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 76us (17351us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303002.413982:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 618 00000100:00100000:0.0:1789303002.415495:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 663 00000100:00100000:0.0:1789303002.415531:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350619648:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303002.417611:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c5c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350619648:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 2062us (21005us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303002.417629:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 663 00010000:00010000:2.0:1789303002.765641:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303003.427681:0:25577:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303003.428112:0:25577:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303003.430178:0:25577:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303003.430203:0:25577:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303003.790053:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789303004.814555:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303005.549047:0:25629:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303005.549058:0:25629:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303005.554007:0:25629:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303005.554029:0:25629:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303005.838717:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303006.221171:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371215360:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.221282:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303006.221359:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02321f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371215872:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.221390:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303006.221419:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789303006.221425:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02322680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371215232:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.221456:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303006.221490:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371215744:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.221537:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303006.221576:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371215488:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.221601:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303006.221618:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789303006.221633:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371215616:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.221655:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303006.221691:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371215360 00000100:00100000:1.0:1789303006.221700:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371215360 00000100:00100000:1.0:1789303006.221754:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371215744 00000100:00100000:1.0:1789303006.221758:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371215744 00000100:00100000:1.0:1789303006.221783:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371215488 00000100:00100000:1.0:1789303006.221786:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371215488 00000100:00100000:1.0:1789303006.221803:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 619 00000100:00100000:1.0:1789303006.221813:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371215360:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222184:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371215360:12345-0@lo:400:kworker.0 Request processed in 370us (900us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303006.222192:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 619 00000100:00100000:1.0:1789303006.222216:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 620 00000100:00100000:1.0:1789303006.222223:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371215744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222255:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f11de5f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371215744:12345-0@lo:400:kworker.0 Request processed in 33us (717us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303006.222261:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 620 00000100:00100000:1.0:1789303006.222274:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:1.0:1789303006.222281:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371215488:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222314:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371215488:12345-0@lo:400:kworker.0 Request processed in 34us (711us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303006.222320:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:1.0:1789303006.222349:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371215872 00000100:00100000:1.0:1789303006.222354:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371215872 00000100:00100000:1.0:1789303006.222382:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371215616 00000100:00100000:1.0:1789303006.222384:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371215616 00000100:00100000:1.0:1789303006.222403:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:1.0:1789303006.222409:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371215872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222441:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02323480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371215872:12345-0@lo:400:kworker.0 Request processed in 32us (1051us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303006.222447:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:1.0:1789303006.222456:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1789303006.222463:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371215616:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222489:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fe32680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371215616:12345-0@lo:400:kworker.0 Request processed in 26us (832us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303006.222494:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1789303006.222579:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371215616:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222592:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371215488:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222637:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371215744:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222648:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371215360:0@lo:400:kworker.0 00000100:00100000:1.0:1789303006.222658:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02321f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371215872:0@lo:400:kworker.0 00000100:00080000:2.0:1789303006.223145:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303006.223151:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303006.223154:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303006.223468:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371215232 00000100:00100000:3.0:1789303006.223487:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371215232 00000100:00100000:3.0:1789303006.223576:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 667 00000100:00100000:3.0:1789303006.223586:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371215232:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303006.223667:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371215232:12345-0@lo:400:kworker.0 Request processed in 84us (2209us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303006.223674:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 667 00000100:00100000:3.0:1789303006.223732:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02322680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371215232:0@lo:400:kworker.0 00000100:00100000:2.0:1789303006.736969:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303006.742280:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371216000:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303006.742395:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303006.742675:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216000 00000100:00100000:1.0:1789303006.742687:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216000 00000100:00100000:1.0:1789303006.742763:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:1.0:1789303006.742775:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371216000:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303006.742827:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303006.742891:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371216000:12345-0@lo:13:osp-pre-1-0.0 Request processed in 118us (496us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303006.742903:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1789303006.746671:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371216000:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1789303006.861648:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303007.248475:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789303007.248537:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371216128:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303007.248644:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303007.248815:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216128 00000100:00100000:0.0:1789303007.248826:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216128 00000100:00100000:0.0:1789303007.249419:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 265 00000100:00100000:0.0:1789303007.249434:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371216128:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789303007.249486:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789303007.251064:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02322680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371216128:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1623us (2408us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303007.251078:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 265 00000100:00100000:0.0:1789303007.251180:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120ca680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371216128:0@lo:13:osp-pre-0-0.0 00000040:00080000:3.0:1789303007.495009:0:25681:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303007.495023:0:25681:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303007.496670:0:25681:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303007.496693:0:25681:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789303007.506771:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303007.506826:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303007.506849:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303007.506882:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303007.506903:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303007.506922:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303007.506950:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303007.506985:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620288 00000100:00100000:2.0:1789303007.506998:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620288 00000100:00100000:2.0:1789303007.507062:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620544 00000100:00100000:2.0:1789303007.507064:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620544 00000100:00100000:2.0:1789303007.507085:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620160 00000100:00100000:2.0:1789303007.507087:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620160 00000100:00100000:2.0:1789303007.507100:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620672 00000100:00100000:2.0:1789303007.507102:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620672 00000100:00100000:2.0:1789303007.507121:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 666 00000100:00100000:2.0:1789303007.507133:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350620288:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303007.507226:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114a300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350620288:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 94us (398us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303007.507234:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 666 00000100:00100000:2.0:1789303007.507252:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 667 00000100:00100000:2.0:1789303007.507258:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350620544:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303007.507281:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350620544:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (431us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303007.507288:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 667 00000100:00100000:2.0:1789303007.507299:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00100000:2.0:1789303007.507306:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350620160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303007.507327:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01149500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350620160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (424us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303007.507333:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00100000:2.0:1789303007.507343:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 669 00000100:00100000:2.0:1789303007.507348:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350620672:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303007.507370:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350620672:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303007.507375:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 669 00000100:00100000:2.0:1789303007.507398:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350619904 00000100:00100000:2.0:1789303007.507402:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350619904 00000100:00100000:2.0:1789303007.507428:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00100000:2.0:1789303007.507435:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350619904:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303007.507463:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350619904:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (580us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303007.507469:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00080000:2.0:1789303007.507522:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303007.507527:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789303007.509044:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620032 00000100:00100000:1.0:1789303007.509058:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620032 00000100:00100000:1.0:1789303007.509128:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620416 00000100:00100000:1.0:1789303007.509131:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620416 00000100:00100000:1.0:1789303007.509160:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 622 00000100:00100000:1.0:1789303007.509169:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350620032:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303007.509247:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0114b800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350620032:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 79us (2491us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303007.509254:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 622 00000100:00100000:1.0:1789303007.509269:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 623 00000100:00100000:1.0:1789303007.509275:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350620416:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303007.509299:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01148380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350620416:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (2377us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303007.509305:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 623 00000100:00080000:2.0:1789303007.510142:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789303007.887258:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789303008.910532:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303009.503613:0:25732:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303009.503625:0:25732:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303009.505979:0:25732:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303009.506002:0:25732:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303009.932766:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303010.509445:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303010.509456:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303010.509996:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303010.510010:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303010.960962:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303011.341798:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01279500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371216640:0@lo:400:kworker.0 00000100:00100000:0.0:1789303011.341798:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01279f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371216256:0@lo:400:kworker.0 00000100:00100000:2.0:1789303011.341954:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01279880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371216384:0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.342790:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303011.342886:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789303011.342898:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371216512:0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.343132:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303011.343201:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216640 00000100:00100000:3.0:1789303011.343212:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216640 00000100:00100000:1.0:1789303011.343325:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216512 00000100:00100000:1.0:1789303011.343334:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216512 00000100:00100000:1.0:1789303011.343390:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:1.0:1789303011.343401:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01278000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371216512:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303011.343483:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01278000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371216512:12345-0@lo:400:kworker.0 Request processed in 83us (350us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303011.343491:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:1.0:1789303011.343568:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127a680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371216512:0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.344696:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:3.0:1789303011.344714:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371216640:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.344793:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371216640:12345-0@lo:400:kworker.0 Request processed in 87us (2008us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303011.344800:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00080000:3.0:1789303011.344833:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789303011.344839:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789303011.345013:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01279500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371216640:0@lo:400:kworker.0 00000100:00100000:0.0:1789303011.345162:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303011.345266:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303011.345323:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e01278e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371216896:0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.345355:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303011.345385:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216896 00000100:00100000:3.0:1789303011.345393:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216896 00000100:00100000:3.0:1789303011.345443:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:3.0:1789303011.345451:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371216896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.345518:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127a680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371216896:12345-0@lo:400:kworker.0 Request processed in 51us (145us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303011.345525:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00000100:00080000:3.0:1789303011.345545:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789303011.345563:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01278e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371216896:0@lo:400:kworker.0 00000100:00100000:1.0:1789303011.345944:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216256 00000100:00100000:1.0:1789303011.345950:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216256 00000100:00100000:1.0:1789303011.346050:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 669 00000100:00100000:1.0:1789303011.346123:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371216256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303011.346177:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371216256:12345-0@lo:400:kworker.0 Request processed in 118us (1015us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303011.346184:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 669 00000100:00100000:2.0:1789303011.347061:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216384 00000100:00100000:2.0:1789303011.347071:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216384 00000100:00100000:2.0:1789303011.347133:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:2.0:1789303011.347145:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371216384:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303011.347229:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371216384:12345-0@lo:400:kworker.0 Request processed in 88us (1962us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303011.347237:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:0.0:1789303011.347280:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01279f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371216256:0@lo:400:kworker.0 00000100:00100000:0.0:1789303011.347294:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0127ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371216768:0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.347358:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e01279880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371216384:0@lo:400:kworker.0 00000100:00100000:0.0:1789303011.347399:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303011.347447:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371216768 00000100:00100000:3.0:1789303011.347451:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371216768 00000100:00100000:3.0:1789303011.347729:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:3.0:1789303011.347737:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371216768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303011.347782:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120ca680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371216768:12345-0@lo:400:kworker.0 Request processed in 46us (383us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303011.347789:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:3.0:1789303011.347986:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0127ad80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371216768:0@lo:400:kworker.0 00000100:00080000:0.0:1789303011.350894:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:0.0:1789303011.582840:0:25783:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303011.582855:0:25783:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303011.584560:0:25783:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303011.584582:0:25783:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789303011.852706:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371217024:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303011.852812:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303011.852912:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217024 00000100:00100000:2.0:1789303011.852922:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217024 00000100:00100000:2.0:1789303011.852981:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:2.0:1789303011.852993:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371217024:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789303011.853061:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789303011.853119:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371217024:12345-0@lo:13:osp-pre-1-0.0 Request processed in 128us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303011.853127:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:2.0:1789303011.853185:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371217024:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789303011.980736:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303012.365065:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371217152:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303012.365158:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303012.373690:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217152 00000100:00100000:0.0:1789303012.373715:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217152 00000100:00100000:0.0:1789303012.373810:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 267 00000100:00100000:0.0:1789303012.373823:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371217152:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789303012.373879:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789303012.373996:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e02321f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371217152:12345-0@lo:13:osp-pre-0-0.0 Request processed in 176us (8837us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303012.374010:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 267 00000100:00100000:1.0:1789303012.377009:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f120caa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371217152:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303012.622031:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303012.622086:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303012.622122:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303012.622150:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303012.622171:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303012.622192:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303012.622217:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303012.622264:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621184 00000100:00100000:3.0:1789303012.622277:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621184 00000100:00100000:3.0:1789303012.622345:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621440 00000100:00100000:3.0:1789303012.622348:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621440 00000100:00100000:3.0:1789303012.622365:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621056 00000100:00100000:3.0:1789303012.622368:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621056 00000100:00100000:3.0:1789303012.622381:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621568 00000100:00100000:3.0:1789303012.622383:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621568 00000100:00100000:3.0:1789303012.622399:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:3.0:1789303012.622411:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350621184:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303012.622471:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620800 00000100:00100000:2.0:1789303012.622480:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620800 00000100:00100000:1.0:1789303012.622543:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350620928 00000100:00100000:3.0:1789303012.622553:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350621184:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 142us (464us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303012.622561:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350620928 00000100:00100000:2.0:1789303012.622568:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 670 00000100:00100000:3.0:1789303012.622574:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:2.0:1789303012.622589:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350620800:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303012.622607:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 673 00000100:00100000:3.0:1789303012.622617:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01278e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350621440:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303012.622628:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621312 00000100:00100000:2.0:1789303012.622630:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350620800:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 51us (476us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303012.622634:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621312 00000100:00100000:2.0:1789303012.622640:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 670 00000100:00100000:3.0:1789303012.622667:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01278e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350621440:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 49us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303012.622675:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 627 00000100:00100000:3.0:1789303012.622680:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 673 00000100:00100000:1.0:1789303012.622689:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0127ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350620928:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303012.622699:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 674 00000100:00100000:3.0:1789303012.622710:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e01279f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350621056:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303012.622732:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0127ad80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350620928:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 46us (717us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303012.622744:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 627 00000100:00100000:1.0:1789303012.622774:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 628 00000100:00100000:1.0:1789303012.622781:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350621312:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303012.622791:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 675 00000100:00100000:0.0:1789303012.622814:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350621568:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:2.0:1789303012.631611:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789303012.631625:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303012.631629:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789303012.635660:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e01279f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350621056:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 12936us (13470us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303012.635678:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 674 00000100:00100000:1.0:1789303012.636588:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350621312:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 13784us (14371us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303012.636603:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 628 00000100:00100000:0.0:1789303012.641976:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1db46300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350621568:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 19154us (19736us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303012.641995:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 675 00010000:00010000:1.0:1789303013.004770:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303013.904198:0:25834:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303013.904214:0:25834:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303013.907176:0:25834:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303013.907207:0:25834:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303014.029578:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789303015.054703:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303015.980482:0:25886:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303015.980499:0:25886:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303015.982233:0:25886:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303015.982252:0:25886:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303016.076658:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303016.462447:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371217664:0@lo:400:kworker.0 00000100:00100000:1.0:1789303016.462529:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303016.462599:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789303016.462606:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371217536:0@lo:400:kworker.0 00000100:00100000:1.0:1789303016.462641:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303016.462686:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217664 00000100:00100000:1.0:1789303016.462704:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217664 00000100:00100000:1.0:1789303016.462757:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:1.0:1789303016.462765:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371217664:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303016.462815:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371217664:12345-0@lo:400:kworker.0 Request processed in 51us (286us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303016.462821:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:1.0:1789303016.462862:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371217664:0@lo:400:kworker.0 00000100:00080000:1.0:1789303016.462889:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789303016.462900:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217536 00000100:00100000:1.0:1789303016.462903:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217536 00000100:00100000:1.0:1789303016.462933:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:1.0:1789303016.462940:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371217536:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303016.462974:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371217536:12345-0@lo:400:kworker.0 Request processed in 34us (332us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303016.462980:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00080000:1.0:1789303016.462993:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303016.463008:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02322a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371217536:0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463240:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371217280:0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463275:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303016.463308:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371217792:0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463330:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303016.463349:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1789303016.463355:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371217408:0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463374:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303016.463390:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b87b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371217920:0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463411:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303016.463443:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217792 00000100:00100000:3.0:1789303016.463448:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217792 00000100:00100000:3.0:1789303016.463478:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217408 00000100:00100000:3.0:1789303016.463481:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217408 00000100:00100000:3.0:1789303016.463499:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:3.0:1789303016.463528:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371217792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463575:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c90000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371217792:12345-0@lo:400:kworker.0 Request processed in 48us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303016.463582:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:3.0:1789303016.463594:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:3.0:1789303016.463601:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371217408:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463638:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f41c92300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371217408:12345-0@lo:400:kworker.0 Request processed in 38us (263us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303016.463645:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00080000:3.0:1789303016.463665:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303016.463683:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02320700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371217408:0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463703:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371217792:0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463730:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217920 00000100:00100000:3.0:1789303016.463734:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217920 00000100:00100000:3.0:1789303016.463762:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:3.0:1789303016.463768:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f027b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371217920:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303016.463800:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f027b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371217920:12345-0@lo:400:kworker.0 Request processed in 32us (388us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303016.463805:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00080000:3.0:1789303016.463817:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789303016.463831:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b87b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371217920:0@lo:400:kworker.0 00000100:00100000:1.0:1789303016.464329:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371217280 00000100:00100000:1.0:1789303016.464335:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371217280 00000100:00100000:1.0:1789303016.464376:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:1.0:1789303016.464383:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371217280:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303016.464426:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371217280:12345-0@lo:400:kworker.0 Request processed in 43us (1150us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303016.464433:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00000100:00100000:3.0:1789303016.466921:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e02323800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371217280:0@lo:400:kworker.0 00000100:00100000:1.0:1789303016.975111:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371218048:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789303016.975196:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303016.975423:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218048 00000100:00100000:0.0:1789303016.975431:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218048 00000100:00100000:0.0:1789303016.975475:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:0.0:1789303016.975484:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371218048:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789303016.975540:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789303016.975604:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371218048:12345-0@lo:13:osp-pre-1-0.0 Request processed in 119us (406us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303016.975610:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:1.0:1789303016.976280:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371218048:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303017.103747:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303017.486407:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789303017.493644:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371218176:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303017.493749:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303017.493944:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218176 00000100:00100000:2.0:1789303017.493955:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218176 00000100:00100000:2.0:1789303017.494027:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 269 00000100:00100000:2.0:1789303017.494038:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371218176:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303017.494087:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303017.494141:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120c9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371218176:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303017.494148:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 269 00000100:00100000:0.0:1789303017.494363:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371218176:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303017.742847:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303017.742990:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621696 00000100:00100000:0.0:1789303017.743004:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621696 00000100:00100000:0.0:1789303017.743079:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:0.0:1789303017.743091:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350621696:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303017.743185:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350621696:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 96us (355us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303017.743194:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:0.0:1789303017.743955:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303017.744229:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621824 00000100:00100000:3.0:1789303017.744239:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621824 00000100:00100000:3.0:1789303017.744312:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 632 00000100:00100000:3.0:1789303017.744323:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350621824:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303017.744387:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f120caa00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350621824:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 67us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303017.744395:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 632 00000100:00080000:0.0:1789303017.744450:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303017.757000:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303017.757090:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350621952 00000100:00100000:2.0:1789303017.757104:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350621952 00000100:00100000:0.0:1789303017.757148:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303017.757177:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303017.757184:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622080 00000100:00100000:2.0:1789303017.757187:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622080 00000100:00100000:0.0:1789303017.757200:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303017.757208:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622336 00000100:00100000:2.0:1789303017.757210:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622336 00000100:00100000:2.0:1789303017.757236:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 678 00000100:00100000:2.0:1789303017.757246:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350621952:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303017.757293:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303017.757336:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3fcdd880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350621952:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 92us (351us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303017.757344:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 678 00000100:00100000:0.0:1789303017.757354:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622464 00000100:00100000:0.0:1789303017.757358:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622464 00000100:00100000:2.0:1789303017.757365:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 679 00000100:00100000:2.0:1789303017.757370:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350622080:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303017.757380:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 680 00000100:00100000:0.0:1789303017.757387:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350622336:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303017.757395:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da0700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350622080:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (248us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303017.757413:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da1180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350622336:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (236us total) trans 0 rc 0/0 00000100:00080000:1.0:1789303017.757741:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789303017.757747:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789303017.757854:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 679 00000100:00100000:3.0:1789303017.758416:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622208 00000100:00100000:3.0:1789303017.758426:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622208 00000100:00100000:3.0:1789303017.758485:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 633 00000100:00100000:3.0:1789303017.758495:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350622208:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303017.758992:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 680 00000100:00100000:0.0:1789303017.759243:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 681 00000100:00100000:0.0:1789303017.759257:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350622464:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303017.759340:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350622464:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 88us (2046us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303017.759347:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 681 00000100:00100000:3.0:1789303017.759410:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da3480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350622208:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 916us (2207us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303017.759419:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 633 00000100:00080000:3.0:1789303017.759468:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000040:00080000:2.0:1789303018.053467:0:25937:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303018.053481:0:25937:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303018.054368:0:25937:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303018.054384:0:25937:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789303018.127218:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789303019.149551:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303020.075227:0:25988:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303020.075240:0:25988:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303020.077087:0:25988:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303020.077108:0:25988:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303020.174845:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303020.748889:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303020.748903:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303020.749422:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303020.749435:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789303021.198646:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789303021.581437:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371218304:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.581545:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303021.581590:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371218816:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.581620:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303021.581644:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1789303021.581650:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371218432:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.581673:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303021.581692:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371218944:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.581715:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303021.581745:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371218560:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.581768:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303021.581787:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789303021.581793:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371218688:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.581815:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303021.581848:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218304 00000100:00100000:2.0:1789303021.581855:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218304 00000100:00100000:2.0:1789303021.581896:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 673 00000100:00100000:2.0:1789303021.581905:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371218304:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.581952:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371218304:12345-0@lo:400:kworker.0 Request processed in 49us (407us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303021.581959:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 673 00000100:00100000:2.0:1789303021.581990:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218816 00000100:00100000:2.0:1789303021.581994:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218816 00000100:00100000:2.0:1789303021.582026:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218432 00000100:00100000:2.0:1789303021.582029:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218432 00000100:00100000:2.0:1789303021.582046:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218560 00000100:00100000:2.0:1789303021.582049:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218560 00000100:00100000:2.0:1789303021.582065:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1789303021.582071:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371218816:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582102:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371218816:12345-0@lo:400:kworker.0 Request processed in 32us (482us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303021.582109:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1789303021.582121:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:2.0:1789303021.582127:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04cea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371218432:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582153:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04cea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371218432:12345-0@lo:400:kworker.0 Request processed in 27us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303021.582159:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:2.0:1789303021.582170:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:2.0:1789303021.582176:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371218560:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582206:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371218560:12345-0@lo:400:kworker.0 Request processed in 31us (437us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303021.582212:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:2.0:1789303021.582230:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218944 00000100:00100000:2.0:1789303021.582234:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218944 00000100:00100000:2.0:1789303021.582259:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371218688 00000100:00100000:2.0:1789303021.582262:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371218688 00000100:00100000:2.0:1789303021.582281:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:2.0:1789303021.582288:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371218944:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582320:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371218944:12345-0@lo:400:kworker.0 Request processed in 33us (604us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303021.582327:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:2.0:1789303021.582338:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00000100:00100000:2.0:1789303021.582344:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ceb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371218688:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582371:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ceb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371218688:12345-0@lo:400:kworker.0 Request processed in 27us (554us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303021.582376:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00000100:00080000:2.0:1789303021.582401:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303021.582404:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303021.582407:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789303021.582447:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371218688:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582457:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371218560:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582486:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371218432:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582496:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371218944:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582546:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371218304:0@lo:400:kworker.0 00000100:00100000:2.0:1789303021.582556:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbe300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371218816:0@lo:400:kworker.0 00000100:00100000:1.0:1789303022.093461:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371219072:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789303022.093558:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303022.093686:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219072 00000100:00100000:2.0:1789303022.093695:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219072 00000100:00100000:2.0:1789303022.093763:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:2.0:1789303022.093786:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371219072:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789303022.093822:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789303022.093890:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b86680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371219072:12345-0@lo:13:osp-pre-1-0.0 Request processed in 117us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303022.093897:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:1.0:1789303022.094010:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371219072:0@lo:13:osp-pre-1-0.0 00000040:00080000:1.0:1789303022.094677:0:26040:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303022.094683:0:26040:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303022.095337:0:26040:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303022.095351:0:26040:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303022.221121:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303022.605862:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371219200:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303022.605929:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303022.606056:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219200 00000100:00100000:0.0:1789303022.606066:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219200 00000100:00100000:0.0:1789303022.606120:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 271 00000100:00100000:0.0:1789303022.606130:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371219200:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1789303022.606167:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:0.0:1789303022.606234:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f024e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371219200:12345-0@lo:13:osp-pre-0-0.0 Request processed in 105us (304us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303022.606241:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 271 00000100:00100000:3.0:1789303022.606326:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371219200:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303022.860205:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303022.860248:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303022.860283:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303022.860318:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303022.860337:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303022.860362:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303022.860415:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303022.860444:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350623104 00000100:00100000:1.0:1789303022.860453:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350623104 00000100:00100000:1.0:1789303022.860554:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622720 00000100:00100000:1.0:1789303022.860558:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622720 00000100:00100000:1.0:1789303022.860590:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 637 00000100:00100000:1.0:1789303022.860600:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350623104:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303022.860788:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 638 00000100:00100000:0.0:1789303022.860798:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350622720:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303022.860835:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350622720:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 40us (496us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303022.860842:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 638 00000100:00100000:1.0:1789303022.861029:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350623104:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 430us (744us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303022.861036:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 637 00000100:00100000:2.0:1789303022.861183:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622848 00000100:00100000:2.0:1789303022.861188:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622848 00000100:00100000:2.0:1789303022.861230:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350623360 00000100:00100000:2.0:1789303022.861234:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350623360 00000100:00100000:2.0:1789303022.861257:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622976 00000100:00100000:2.0:1789303022.861260:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622976 00000100:00100000:2.0:1789303022.861274:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350623232 00000100:00100000:2.0:1789303022.861276:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350623232 00000100:00100000:2.0:1789303022.861290:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 684 00000100:00100000:2.0:1789303022.861298:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350622848:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303022.861334:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b86a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350622848:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (1085us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303022.861341:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 684 00000100:00100000:2.0:1789303022.861354:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 685 00000100:00100000:2.0:1789303022.861360:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350623360:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303022.861383:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350623360:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (1065us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303022.861389:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 685 00000100:00100000:2.0:1789303022.861401:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 686 00000100:00100000:2.0:1789303022.861407:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350622976:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303022.861430:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350622976:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (1068us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303022.861435:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 686 00000100:00100000:2.0:1789303022.861445:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 687 00000100:00100000:2.0:1789303022.861451:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350623232:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303022.861473:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350623232:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (1059us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303022.861478:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 687 00000100:00080000:2.0:1789303022.861534:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303022.861538:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303022.861540:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303022.861895:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350622592 00000100:00100000:3.0:1789303022.861901:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350622592 00000100:00100000:3.0:1789303022.861982:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 674 00000100:00100000:3.0:1789303022.861990:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350622592:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303022.862021:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f2050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350622592:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (1825us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303022.862028:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 674 00010000:00010000:2.0:1789303023.244730:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303023.938908:0:26091:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303023.938919:0:26091:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303023.940487:0:26091:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303023.941289:0:26091:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789303024.270833:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1789303025.292738:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303025.867846:0:26142:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303025.867858:0:26142:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303025.868467:0:26142:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303025.868481:0:26142:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303026.320399:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303026.701522:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b85c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371219584:0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.701614:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303026.701694:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789303026.701701:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371219712:0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.701732:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303026.701768:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371219328:0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.701782:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219584 00000100:00100000:1.0:1789303026.701792:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219584 00000100:00100000:0.0:1789303026.701808:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303026.701847:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371219840:0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.701854:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 639 00000100:00100000:1.0:1789303026.701864:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371219584:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303026.701870:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303026.701890:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789303026.701895:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371219456:0@lo:400:kworker.0 00000100:00100000:0.0:1789303026.701914:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303026.701922:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371219584:12345-0@lo:400:kworker.0 Request processed in 60us (309us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303026.701926:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371219968:0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.701930:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 639 00000100:00100000:0.0:1789303026.701945:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303026.701946:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219840 00000100:00100000:1.0:1789303026.701949:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219840 00000100:00100000:1.0:1789303026.701966:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219456 00000100:00100000:1.0:1789303026.701969:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219456 00000100:00100000:0.0:1789303026.701971:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:0.0:1789303026.701977:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371219840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.701982:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:1.0:1789303026.701987:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371219456:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303026.702062:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219712 00000100:00100000:2.0:1789303026.702067:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219712 00000100:00100000:2.0:1789303026.702101:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219968 00000100:00100000:2.0:1789303026.702105:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219968 00000100:00100000:2.0:1789303026.702129:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 688 00000100:00100000:2.0:1789303026.702137:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371219712:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303026.702175:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371219712:12345-0@lo:400:kworker.0 Request processed in 38us (441us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303026.702183:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 688 00000100:00100000:2.0:1789303026.702196:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 689 00000100:00100000:2.0:1789303026.702202:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371219968:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303026.702242:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da1f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371219840:12345-0@lo:400:kworker.0 Request processed in 264us (370us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303026.702249:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00100000:3.0:1789303026.702610:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371219328 00000100:00100000:3.0:1789303026.702616:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371219328 00000100:00100000:3.0:1789303026.702719:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:3.0:1789303026.702727:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371219328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.704749:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da3800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371219456:12345-0@lo:400:kworker.0 Request processed in 2750us (2822us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303026.704764:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:1.0:1789303026.704864:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371219712:0@lo:400:kworker.0 00000100:00100000:1.0:1789303026.704881:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b85c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371219584:0@lo:400:kworker.0 00000100:00100000:2.0:1789303026.704992:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371219968:12345-0@lo:400:kworker.0 Request processed in 2787us (3042us total) trans 0 rc 0/0 00000100:00080000:1.0:1789303026.705094:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789303026.705100:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789303026.705104:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303026.706740:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371219456:0@lo:400:kworker.0 00000100:00100000:0.0:1789303026.706773:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371219968:0@lo:400:kworker.0 00000100:00100000:2.0:1789303026.706977:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 689 00000100:00100000:3.0:1789303026.707442:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371219328:12345-0@lo:400:kworker.0 Request processed in 4711us (5628us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303026.707465:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00000100:00100000:0.0:1789303026.708745:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371219328:0@lo:400:kworker.0 00000100:00100000:0.0:1789303026.708776:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b84700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371219840:0@lo:400:kworker.0 00000100:00100000:3.0:1789303027.213063:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b85c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371220096:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789303027.213184:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303027.213394:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220096 00000100:00100000:1.0:1789303027.213404:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220096 00000100:00100000:1.0:1789303027.213463:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:1.0:1789303027.213475:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371220096:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303027.213982:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303027.214076:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f026d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371220096:12345-0@lo:13:osp-pre-1-0.0 Request processed in 602us (891us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303027.214086:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00000100:00100000:3.0:1789303027.214215:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b85c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371220096:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789303027.345222:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303027.728439:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303027.728467:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04cebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371220224:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303027.728731:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303027.729210:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220224 00000100:00100000:2.0:1789303027.729220:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220224 00000100:00100000:2.0:1789303027.729280:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 273 00000100:00100000:2.0:1789303027.729290:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371220224:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303027.729334:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303027.729397:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d57100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371220224:12345-0@lo:13:osp-pre-0-0.0 Request processed in 108us (668us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303027.729404:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 273 00000100:00100000:2.0:1789303027.729836:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04cebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371220224:0@lo:13:osp-pre-0-0.0 00000040:00080000:3.0:1789303027.802867:0:26193:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303027.802879:0:26193:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303027.804793:0:26193:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303027.804816:0:26193:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789303027.986256:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303027.986300:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303027.986327:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303027.986346:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303027.986364:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303027.986380:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303027.986401:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303027.986432:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350623744 00000100:00100000:3.0:1789303027.986444:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350623744 00000100:00100000:3.0:1789303027.986537:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624256 00000100:00100000:3.0:1789303027.986540:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624256 00000100:00100000:3.0:1789303027.986575:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350623872 00000100:00100000:3.0:1789303027.986578:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350623872 00000100:00100000:3.0:1789303027.986592:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624128 00000100:00100000:3.0:1789303027.986594:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624128 00000100:00100000:3.0:1789303027.986615:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 690 00000100:00100000:2.0:1789303027.987159:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350623488 00000100:00100000:2.0:1789303027.987166:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350623488 00000100:00100000:2.0:1789303027.987202:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 676 00000100:00100000:2.0:1789303027.987213:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350623488:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303027.987321:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f1850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350623488:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 110us (1079us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303027.987329:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 676 00000100:00100000:2.0:1789303027.989487:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624000 00000100:00100000:3.0:1789303027.990056:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350623744:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303027.990158:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350623744:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 120us (3855us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303027.990217:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624000 00000100:00100000:2.0:1789303027.990285:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350623616 00000100:00100000:2.0:1789303027.990290:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350623616 00000100:00100000:2.0:1789303027.990316:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00100000:2.0:1789303027.990324:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350624000:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303027.990356:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350624000:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (4029us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303027.990367:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 642 00000100:00100000:2.0:1789303027.990384:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 643 00000100:00100000:2.0:1789303027.990390:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350623616:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303027.990408:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350623616:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (4062us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303027.990414:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 643 00000100:00100000:1.0:1789303027.990779:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 691 00000100:00100000:1.0:1789303027.990793:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350624256:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303027.990831:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350624256:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (4465us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303027.990837:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 691 00000100:00100000:1.0:1789303027.990859:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:1.0:1789303027.990866:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350623872:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303027.990892:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350623872:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (4512us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303027.990897:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:1.0:1789303027.990910:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 693 00000100:00100000:1.0:1789303027.990917:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350624128:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303027.990970:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350624128:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 53us (4567us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303027.990978:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 693 00000100:00080000:1.0:1789303027.991011:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789303027.991016:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789303027.991019:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789303027.993554:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 690 00010000:00010000:1.0:1789303028.364766:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303029.396002:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303029.963081:0:26245:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303029.963092:0:26245:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303029.963641:0:26245:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303029.963652:0:26245:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789303030.412895:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303030.989182:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303030.989193:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303030.991348:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303030.991372:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303031.436672:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789303031.826304:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371220352:0@lo:400:kworker.0 00000100:00100000:0.0:1789303031.826377:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303031.827944:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03da1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371220736:0@lo:400:kworker.0 00000100:00100000:0.0:1789303031.827996:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303031.828027:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789303031.828033:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03da2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371220608:0@lo:400:kworker.0 00000100:00100000:0.0:1789303031.828058:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303031.828088:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220736 00000100:00100000:0.0:1789303031.828096:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220736 00000100:00100000:0.0:1789303031.828224:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:0.0:1789303031.828235:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371220736:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303031.828284:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da2d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371220736:12345-0@lo:400:kworker.0 Request processed in 52us (288us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303031.828292:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:0.0:1789303031.828314:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220608 00000100:00100000:0.0:1789303031.828318:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220608 00000100:00100000:1.0:1789303031.828363:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03da0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371220864:0@lo:400:kworker.0 00000100:00100000:1.0:1789303031.828392:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303031.828412:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1789303031.828417:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03da2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371220480:0@lo:400:kworker.0 00000100:00100000:1.0:1789303031.828435:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303031.828462:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03da0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371220992:0@lo:400:kworker.0 00000100:00100000:1.0:1789303031.828481:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303031.828524:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220864 00000100:00100000:1.0:1789303031.828527:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220864 00000100:00100000:2.0:1789303031.829572:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220480 00000100:00100000:2.0:1789303031.829582:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220480 00000100:00100000:0.0:1789303031.834808:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:0.0:1789303031.834830:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371220608:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303031.834915:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371220608:12345-0@lo:400:kworker.0 Request processed in 96us (6856us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303031.834922:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00100000:1.0:1789303031.834994:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:1.0:1789303031.835003:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371220864:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303031.835017:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220352 00000100:00100000:3.0:1789303031.835026:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220352 00000100:00100000:1.0:1789303031.835046:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371220864:12345-0@lo:400:kworker.0 Request processed in 45us (6652us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303031.835051:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:3.0:1789303031.835061:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:3.0:1789303031.835068:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371220352:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303031.835072:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371220992 00000100:00100000:1.0:1789303031.835075:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371220992 00000100:00100000:3.0:1789303031.835102:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f0450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371220352:12345-0@lo:400:kworker.0 Request processed in 34us (8724us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303031.835102:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 695 00000100:00100000:1.0:1789303031.835108:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371220992:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303031.835111:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:1.0:1789303031.835137:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371220992:12345-0@lo:400:kworker.0 Request processed in 30us (6655us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303031.835142:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 695 00000100:00100000:1.0:1789303031.835193:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03da0380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371220992:0@lo:400:kworker.0 00000100:00100000:1.0:1789303031.835205:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03da0a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371220864:0@lo:400:kworker.0 00000100:00100000:1.0:1789303031.835214:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371220352:0@lo:400:kworker.0 00000100:00100000:2.0:1789303031.838556:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:2.0:1789303031.838572:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371220480:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303031.838658:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371220480:12345-0@lo:400:kworker.0 Request processed in 91us (10220us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303031.838665:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:2.0:1789303031.838734:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03da2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371220480:0@lo:400:kworker.0 00000100:00080000:2.0:1789303031.839585:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303031.839592:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789303031.839595:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789303031.841608:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03da2680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371220608:0@lo:400:kworker.0 00000100:00100000:1.0:1789303031.841624:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03da1500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371220736:0@lo:400:kworker.0 00000040:00080000:3.0:1789303031.929354:0:26296:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303031.929363:0:26296:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303031.930103:0:26296:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303031.930114:0:26296:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789303032.335391:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789303032.335416:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371221120:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789303032.335495:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303032.336978:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371221120 00000100:00100000:1.0:1789303032.336991:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371221120 00000100:00100000:1.0:1789303032.337061:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:1.0:1789303032.337071:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371221120:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303032.337112:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303032.337207:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371221120:12345-0@lo:13:osp-pre-1-0.0 Request processed in 137us (1712us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303032.337215:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:2.0:1789303032.337806:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371221120:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303032.460900:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303032.845179:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303032.845202:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04cebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371221248:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303032.845278:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303032.845470:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371221248 00000100:00100000:1.0:1789303032.845487:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371221248 00000100:00100000:1.0:1789303032.845563:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 275 00000100:00100000:1.0:1789303032.845584:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3b2b9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371221248:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789303032.845619:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303032.845687:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3b2b9850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371221248:12345-0@lo:13:osp-pre-0-0.0 Request processed in 106us (409us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303032.845695:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 275 00000100:00100000:3.0:1789303032.846558:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04cebb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371221248:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303033.101497:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303033.101558:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303033.101594:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303033.101613:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303033.101631:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303033.101652:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303033.101675:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303033.101698:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624896 00000100:00100000:0.0:1789303033.101707:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624896 00000100:00100000:0.0:1789303033.101762:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624512 00000100:00100000:0.0:1789303033.101765:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624512 00000100:00100000:0.0:1789303033.101788:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 647 00000100:00100000:0.0:1789303033.101797:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350624896:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303033.101872:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350624896:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 75us (276us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303033.101879:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 647 00000100:00100000:0.0:1789303033.101895:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:0.0:1789303033.101902:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350624512:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303033.101926:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da0e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350624512:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (295us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303033.101933:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 648 00000100:00100000:0.0:1789303033.101949:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624640 00000100:00100000:0.0:1789303033.101953:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624640 00000100:00100000:0.0:1789303033.101974:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625152 00000100:00100000:0.0:1789303033.101976:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625152 00000100:00100000:0.0:1789303033.101992:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624768 00000100:00100000:0.0:1789303033.101994:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624768 00000100:00100000:0.0:1789303033.102006:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 696 00000100:00100000:0.0:1789303033.102024:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350624640:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303033.102048:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350624640:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303033.102054:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 696 00000100:00100000:0.0:1789303033.102063:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625024 00000100:00100000:0.0:1789303033.102066:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625024 00000100:00100000:0.0:1789303033.102079:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 697 00000100:00100000:0.0:1789303033.102085:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350625152:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303033.102104:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da2300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350625152:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 19us (490us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303033.102109:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 697 00000100:00100000:0.0:1789303033.102119:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:0.0:1789303033.102125:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350624768:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303033.102223:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da1500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350624768:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 97us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303033.102231:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:0.0:1789303033.102243:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 699 00000100:00100000:0.0:1789303033.102249:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350625024:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303033.102271:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da2680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350625024:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (596us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303033.102277:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 699 00000100:00080000:0.0:1789303033.102292:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789303033.102296:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303033.102298:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303033.102843:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350624384 00000100:00100000:1.0:1789303033.102850:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350624384 00000100:00100000:1.0:1789303033.102894:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 678 00000100:00100000:1.0:1789303033.102903:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350624384:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303033.102937:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350624384:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (1450us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303033.102944:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 678 00010000:00010000:2.0:1789303033.484694:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303033.843036:0:26347:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303033.843046:0:26347:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303033.854309:0:26347:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303033.854340:0:26347:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303034.508663:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303035.533096:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303035.703319:0:26398:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303035.703329:0:26398:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303035.704097:0:26398:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303035.704106:0:26398:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303036.556749:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789303036.942089:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ceaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371221760:0@lo:400:kworker.0 00000100:00100000:0.0:1789303036.942089:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ceb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371221504:0@lo:400:kworker.0 00000100:00100000:3.0:1789303036.945541:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371221376:0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.947708:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303036.947768:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303036.947779:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371221632:0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.947813:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303036.947845:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371221760 00000100:00100000:2.0:1789303036.947853:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371221760 00000100:00100000:2.0:1789303036.947904:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 700 00000100:00100000:2.0:1789303036.947912:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371221760:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.947970:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371221760:12345-0@lo:400:kworker.0 Request processed in 58us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303036.947977:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 700 00000100:00100000:0.0:1789303036.948002:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303036.948014:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371221632 00000100:00100000:2.0:1789303036.948018:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371221632 00000100:00100000:0.0:1789303036.948029:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371222016:0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.948042:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371221504 00000100:00100000:2.0:1789303036.948044:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371221504 00000100:00100000:0.0:1789303036.948051:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303036.948062:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:2.0:1789303036.948067:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371221632:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303036.948090:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:2.0:1789303036.948096:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371221632:12345-0@lo:400:kworker.0 Request processed in 29us (283us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303036.948097:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371221504:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.948102:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00080000:2.0:1789303036.948117:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789303036.948122:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789303036.948130:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222016 00000100:00100000:2.0:1789303036.948132:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222016 00000100:00100000:2.0:1789303036.948160:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 701 00000100:00100000:2.0:1789303036.948166:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371222016:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.948194:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371222016:12345-0@lo:400:kworker.0 Request processed in 29us (142us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303036.948200:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 701 00000100:00080000:2.0:1789303036.948221:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789303036.948245:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371221632:0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.948255:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ceaa00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371221760:0@lo:400:kworker.0 00000100:00100000:0.0:1789303036.948576:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371221504:12345-0@lo:400:kworker.0 Request processed in 32us (126us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303036.948585:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:0.0:1789303036.948612:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ceb800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371221504:0@lo:400:kworker.0 00000100:00100000:0.0:1789303036.948621:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371222016:0@lo:400:kworker.0 00000100:00100000:3.0:1789303036.950073:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303036.950122:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04cea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371221888:0@lo:400:kworker.0 00000100:00100000:3.0:1789303036.950155:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303036.950243:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371221376 00000100:00100000:2.0:1789303036.950251:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371221376 00000100:00100000:2.0:1789303036.950294:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 679 00000100:00100000:2.0:1789303036.950303:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371221376:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303036.950351:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334f3850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371221376:12345-0@lo:400:kworker.0 Request processed in 50us (281us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303036.950358:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 679 00000100:00100000:1.0:1789303036.950447:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371221888 00000100:00100000:1.0:1789303036.950452:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371221888 00000100:00080000:2.0:1789303036.950775:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303036.950843:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:1.0:1789303036.950852:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d57480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371221888:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303036.950888:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d57480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371221888:12345-0@lo:400:kworker.0 Request processed in 39us (732us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303036.950897:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00100000:3.0:1789303036.951061:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371221376:0@lo:400:kworker.0 00000100:00100000:3.0:1789303036.951075:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04cea680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371221888:0@lo:400:kworker.0 00000100:00100000:2.0:1789303037.463849:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303037.470705:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b85880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371222144:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303037.470776:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303037.471014:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222144 00000100:00100000:1.0:1789303037.471022:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222144 00000100:00100000:1.0:1789303037.471071:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1789303037.471081:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ceaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371222144:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303037.471118:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303037.471178:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ceaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371222144:12345-0@lo:13:osp-pre-1-0.0 Request processed in 98us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303037.471184:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:0.0:1789303037.471271:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b85880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371222144:0@lo:13:osp-pre-1-0.0 00000040:00080000:2.0:1789303037.579064:0:26450:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303037.579079:0:26450:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303037.579669:0:26450:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303037.579679:0:26450:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:1.0:1789303037.584079:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303037.964776:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303037.964801:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371222272:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303037.964877:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303037.964928:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222272 00000100:00100000:1.0:1789303037.964937:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222272 00000100:00100000:1.0:1789303037.964986:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 277 00000100:00100000:2.0:1789303037.964993:0:6698:0:(service.c:3467:ptlrpc_start_thread()) ost_create[0] started 2 min 2 max 16 00000100:00100000:1.0:1789303037.964998:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371222272:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789303037.965006:0:6698:0:(service.c:3526:ptlrpc_start_thread()) starting thread 'll_ost_create00_002' 00002000:00000024:1.0:1789303037.965034:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303037.965100:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d57b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371222272:12345-0@lo:13:osp-pre-0-0.0 Request processed in 104us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303037.965108:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 277 00000100:00100000:2.0:1789303037.965211:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371222272:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303038.220904:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303038.220960:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303038.220988:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303038.221040:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303038.221045:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625408 00000100:00100000:2.0:1789303038.221056:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625408 00000100:00100000:0.0:1789303038.221069:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303038.221095:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303038.221120:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625792 00000100:00100000:2.0:1789303038.221124:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625792 00000100:00100000:0.0:1789303038.221129:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303038.221151:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 652 00000100:00100000:0.0:1789303038.221156:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625280 00000100:00100000:2.0:1789303038.221163:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03da0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350625408:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303038.221164:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625280 00000100:00100000:0.0:1789303038.221198:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 680 00000100:00100000:0.0:1789303038.221207:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350625280:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303038.221239:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03da0380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350625408:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 78us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303038.221245:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350625280:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (203us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303038.221252:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 652 00000100:00100000:0.0:1789303038.221257:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 680 00000100:00100000:2.0:1789303038.221276:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 653 00000100:00100000:2.0:1789303038.221285:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350625792:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303038.221313:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef78380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2347:x1876219350625792:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 30us (217us total) trans 0 rc 0/0 00000100:00080000:0.0:1789303038.221314:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789303038.221323:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 653 00000100:00100000:0.0:1789303038.221328:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625664 00000100:00100000:0.0:1789303038.221334:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625664 00000100:00100000:2.0:1789303038.221348:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625920 00000100:00100000:0.0:1789303038.221364:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350625536 00000100:00100000:2.0:1789303038.221367:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625920 00000100:00100000:0.0:1789303038.221368:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350625536 00000100:00100000:2.0:1789303038.221410:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626048 00000100:00100000:2.0:1789303038.221417:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626048 00000100:00100000:0.0:1789303038.221462:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 702 00000100:00100000:0.0:1789303038.221471:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350625664:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303038.221484:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 703 00000100:00100000:1.0:1789303038.221496:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350625920:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303038.221528:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7cb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350625664:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 57us (566us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303038.221537:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 702 00000100:00100000:1.0:1789303038.221549:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f7c8380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350625920:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 55us (558us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303038.221555:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 704 00000100:00100000:1.0:1789303038.221562:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 703 00000100:00100000:0.0:1789303038.221567:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350625536:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303038.221585:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 705 00000100:00100000:1.0:1789303038.221594:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350626048:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303038.221601:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef33b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350625536:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (530us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303038.221609:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 704 00000100:00100000:1.0:1789303038.221627:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f70e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350626048:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (498us total) trans 0 rc 0/0 00000100:00080000:0.0:1789303038.221629:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303038.221634:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789303038.221638:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 705 00010000:00010000:3.0:1789303038.604816:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303039.403731:0:26502:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303039.403739:0:26502:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303039.404260:0:26502:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303039.404269:0:26502:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303039.628695:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303040.204776:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303040.204787:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303040.205337:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303040.205347:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789303040.653601:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303041.178810:0:26553:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303041.178820:0:26553:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303041.179368:0:26553:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303041.179378:0:26553:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303041.676727:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303042.061039:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b85880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371222784:0@lo:400:kworker.0 00000100:00100000:1.0:1789303042.061105:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303042.061168:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789303042.061174:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371222656:0@lo:400:kworker.0 00000100:00100000:1.0:1789303042.061200:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303042.061234:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222784 00000100:00100000:1.0:1789303042.061241:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222784 00000100:00100000:1.0:1789303042.061298:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 706 00000100:00100000:1.0:1789303042.061307:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371222784:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303042.061358:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e011c4e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371222784:12345-0@lo:400:kworker.0 Request processed in 53us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303042.061364:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 706 00000100:00100000:1.0:1789303042.061423:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b85880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371222784:0@lo:400:kworker.0 00000100:00100000:2.0:1789303042.061575:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222656 00000100:00100000:2.0:1789303042.061580:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222656 00000100:00100000:2.0:1789303042.061610:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:2.0:1789303042.061619:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371222656:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303042.061668:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371222656:12345-0@lo:400:kworker.0 Request processed in 52us (466us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303042.061677:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:0.0:1789303042.061820:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371222528:0@lo:400:kworker.0 00000100:00100000:0.0:1789303042.061853:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303042.061892:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e011c5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371223040:0@lo:400:kworker.0 00000100:00100000:0.0:1789303042.061933:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303042.061953:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789303042.061959:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371222400:0@lo:400:kworker.0 00000100:00100000:0.0:1789303042.061986:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303042.062005:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371222912:0@lo:400:kworker.0 00000100:00100000:0.0:1789303042.062024:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789303042.062044:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789303042.062048:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789303042.062056:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223040 00000100:00100000:0.0:1789303042.062060:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223040 00000100:00100000:0.0:1789303042.062088:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 707 00000100:00100000:0.0:1789303042.062094:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f0859b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371223040:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303042.062128:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f0859b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371223040:12345-0@lo:400:kworker.0 Request processed in 33us (193us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303042.062133:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 707 00000100:00080000:0.0:1789303042.062329:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789303042.062349:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e011c5880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371223040:0@lo:400:kworker.0 00000100:00100000:1.0:1789303042.062654:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b84000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371222656:0@lo:400:kworker.0 00000100:00100000:2.0:1789303042.062718:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222528 00000100:00100000:2.0:1789303042.062723:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222528 00000100:00100000:2.0:1789303042.062755:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222912 00000100:00100000:2.0:1789303042.062759:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222912 00000100:00100000:2.0:1789303042.062790:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:2.0:1789303042.062797:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371222528:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303042.062839:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371222528:12345-0@lo:400:kworker.0 Request processed in 42us (985us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303042.062845:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:2.0:1789303042.062857:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:2.0:1789303042.062863:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371222912:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303042.062888:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371222912:12345-0@lo:400:kworker.0 Request processed in 26us (863us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303042.062894:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00100000:2.0:1789303042.062930:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371222912:0@lo:400:kworker.0 00000100:00100000:2.0:1789303042.062940:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b85180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371222528:0@lo:400:kworker.0 00000100:00100000:3.0:1789303042.063070:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371222400 00000100:00100000:3.0:1789303042.063075:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371222400 00000100:00100000:3.0:1789303042.063100:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 681 00000100:00100000:3.0:1789303042.063105:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371222400:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303042.063143:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bec50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371222400:12345-0@lo:400:kworker.0 Request processed in 37us (1156us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303042.063149:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 681 00000100:00100000:3.0:1789303042.063193:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371222400:0@lo:400:kworker.0 00000100:00080000:1.0:1789303042.064960:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303042.572758:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789303042.572785:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371223168:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789303042.572856:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303042.572981:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223168 00000100:00100000:1.0:1789303042.572991:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223168 00000100:00100000:1.0:1789303042.573050:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:1.0:1789303042.573059:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371223168:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303042.573099:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303042.573155:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371223168:12345-0@lo:13:osp-pre-1-0.0 Request processed in 96us (298us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303042.573161:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:0.0:1789303042.573293:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371223168:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789303042.702646:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303043.084825:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371223296:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303043.084903:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303043.084962:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223296 00000100:00100000:2.0:1789303043.084970:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223296 00000100:00100000:2.0:1789303043.085026:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 279 00000100:00100000:2.0:1789303043.085036:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371223296:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303043.085072:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303043.085119:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d57800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371223296:12345-0@lo:13:osp-pre-0-0.0 Request processed in 85us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303043.085130:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 279 00000100:00100000:3.0:1789303043.085414:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371223296:0@lo:13:osp-pre-0-0.0 00000040:00080000:3.0:1789303043.179265:0:26605:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303043.179275:0:26605:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303043.182088:0:26605:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303043.182109:0:26605:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789303043.349784:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303043.349820:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303043.349841:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303043.349866:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303043.349885:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303043.349903:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303043.349947:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303043.349974:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626304 00000100:00100000:3.0:1789303043.349984:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626304 00000100:00100000:3.0:1789303043.350040:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626688 00000100:00100000:3.0:1789303043.350044:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626688 00000100:00100000:3.0:1789303043.350068:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 657 00000100:00100000:3.0:1789303043.350076:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350626304:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303043.350148:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2348:x1876219350626304:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 71us (373us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303043.350155:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 657 00000100:00100000:3.0:1789303043.350173:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 658 00000100:00100000:3.0:1789303043.350180:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350626688:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303043.350203:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350626688:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (299us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303043.350209:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 658 00000100:00100000:3.0:1789303043.350227:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626560 00000100:00100000:3.0:1789303043.350231:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626560 00000100:00100000:3.0:1789303043.350254:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626816 00000100:00100000:3.0:1789303043.350258:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626816 00000100:00100000:3.0:1789303043.350274:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626432 00000100:00100000:3.0:1789303043.350276:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626432 00000100:00100000:3.0:1789303043.350288:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626944 00000100:00100000:3.0:1789303043.350290:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626944 00000100:00100000:3.0:1789303043.350304:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:3.0:1789303043.350310:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350626560:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303043.350337:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d56a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350626560:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303043.350344:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:3.0:1789303043.350356:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 709 00000100:00100000:3.0:1789303043.350362:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350626816:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303043.350403:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d56680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350626816:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 41us (561us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303043.350409:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 709 00000100:00100000:3.0:1789303043.350422:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 710 00000100:00100000:3.0:1789303043.350428:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350626432:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303043.350544:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350626432:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 115us (657us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303043.350550:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 710 00000100:00100000:3.0:1789303043.350563:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 711 00000100:00100000:3.0:1789303043.350569:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350626944:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303043.350594:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350626944:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (647us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303043.350599:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 711 00000100:00100000:3.0:1789303043.350619:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350626176 00000100:00100000:3.0:1789303043.350623:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350626176 00000100:00100000:3.0:1789303043.350646:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 682 00000100:00100000:3.0:1789303043.350651:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350626176:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303043.350672:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350626176:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (806us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303043.350678:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 682 00000100:00080000:3.0:1789303043.350710:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789303043.350714:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789303043.350717:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789303043.724933:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303044.748689:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303044.978210:0:26656:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303044.978219:0:26656:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303044.985138:0:26656:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303044.985156:0:26656:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303045.772769:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303046.805334:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303046.806466:0:26707:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303046.806473:0:26707:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303046.807330:0:26707:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303046.807352:0:26707:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789303047.182297:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789303047.182320:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371223680:0@lo:400:kworker.0 00000100:00100000:3.0:1789303047.182401:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303047.182559:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223680 00000100:00100000:3.0:1789303047.182566:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223680 00000100:00100000:3.0:1789303047.182620:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:3.0:1789303047.182629:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f22317c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371223680:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303047.182681:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f22317c50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371223680:12345-0@lo:400:kworker.0 Request processed in 53us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303047.182688:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:0.0:1789303047.183543:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d56d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371223424:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.183595:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303047.183634:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789303047.183641:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d56300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371223552:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.183670:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303047.183713:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223424 00000100:00100000:0.0:1789303047.183720:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223424 00000100:00100000:0.0:1789303047.183758:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00000100:00100000:0.0:1789303047.183765:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371223424:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.183809:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bdc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371223424:12345-0@lo:400:kworker.0 Request processed in 44us (213us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303047.183815:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00000100:00100000:0.0:1789303047.183842:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223552 00000100:00100000:0.0:1789303047.183846:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223552 00000100:00100000:0.0:1789303047.183870:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:0.0:1789303047.183876:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371223552:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.183906:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371223552:12345-0@lo:400:kworker.0 Request processed in 30us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303047.183912:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00100000:0.0:1789303047.183952:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d56300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371223552:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.183962:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d56d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371223424:0@lo:400:kworker.0 00000100:00080000:0.0:1789303047.184002:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303047.184043:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371223936:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.184066:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303047.184102:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789303047.184107:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371224064:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.184129:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303047.184169:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371223808:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.184189:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303047.184213:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371223680:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.184235:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223936 00000100:00100000:0.0:1789303047.184239:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223936 00000100:00100000:0.0:1789303047.184264:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:0.0:1789303047.184271:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371223936:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.184302:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371223936:12345-0@lo:400:kworker.0 Request processed in 32us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303047.184308:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00080000:0.0:1789303047.184321:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303047.184351:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef41f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371223936:0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.184378:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224064 00000100:00100000:0.0:1789303047.184382:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224064 00000100:00100000:2.0:1789303047.186623:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 712 00000100:00100000:2.0:1789303047.186641:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371224064:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303047.186730:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371224064:12345-0@lo:400:kworker.0 Request processed in 95us (2600us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303047.186738:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 712 00000100:00100000:2.0:1789303047.186793:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371224064:0@lo:400:kworker.0 00000100:00080000:1.0:1789303047.187076:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789303047.187176:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371223808 00000100:00100000:0.0:1789303047.187185:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371223808 00000100:00100000:0.0:1789303047.187278:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 713 00000100:00100000:0.0:1789303047.187286:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371223808:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303047.187335:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371223808:12345-0@lo:400:kworker.0 Request processed in 50us (3144us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303047.187353:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 713 00000100:00100000:0.0:1789303047.187388:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef40380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371223808:0@lo:400:kworker.0 00000100:00080000:1.0:1789303047.193874:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789303047.692979:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789303047.694612:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371224192:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789303047.694689:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303047.694888:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224192 00000100:00100000:3.0:1789303047.694898:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224192 00000100:00100000:3.0:1789303047.694952:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:3.0:1789303047.694960:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371224192:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789303047.694996:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789303047.695041:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371224192:12345-0@lo:13:osp-pre-1-0.0 Request processed in 81us (351us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303047.695049:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1789303047.695719:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371224192:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303047.820774:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789303048.213139:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789303048.217160:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371224320:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789303048.217239:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303048.222784:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224320 00000100:00100000:3.0:1789303048.222801:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224320 00000100:00100000:3.0:1789303048.222914:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 281 00000100:00100000:3.0:1789303048.222925:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04cead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371224320:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789303048.223072:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789303048.223170:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04cead80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371224320:12345-0@lo:13:osp-pre-0-0.0 Request processed in 247us (5930us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303048.223177:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 281 00000100:00100000:1.0:1789303048.223323:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371224320:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303048.462577:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303048.462622:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303048.462652:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303048.462679:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627328 00000100:00100000:0.0:1789303048.462688:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627328 00000100:00100000:0.0:1789303048.462741:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 714 00000100:00100000:0.0:1789303048.462751:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350627328:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303048.462813:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef41f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350627328:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 64us (190us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303048.462820:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 714 00000100:00100000:0.0:1789303048.462859:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627584 00000100:00100000:0.0:1789303048.462863:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627584 00000100:00100000:0.0:1789303048.462891:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00100000:0.0:1789303048.462899:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350627584:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303048.462926:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef43100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350627584:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (272us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303048.462931:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 662 00000100:00080000:0.0:1789303048.462948:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789303048.462952:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789303048.463413:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627072 00000100:00100000:3.0:1789303048.463419:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627072 00000100:00100000:3.0:1789303048.463465:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 684 00000100:00100000:3.0:1789303048.463473:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350627072:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303048.463525:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350627072:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 53us (957us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303048.463531:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 684 00000100:00100000:3.0:1789303048.464083:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303048.464104:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303048.464126:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303048.465044:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627200 00000100:00100000:0.0:1789303048.465054:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627200 00000100:00100000:0.0:1789303048.465114:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 663 00000100:00100000:0.0:1789303048.465123:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350627200:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303048.465188:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350627200:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 66us (1105us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303048.465194:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 663 00000100:00080000:0.0:1789303048.465213:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303048.465614:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627456 00000100:00100000:3.0:1789303048.465621:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627456 00000100:00100000:3.0:1789303048.465658:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627712 00000100:00100000:3.0:1789303048.465661:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627712 00000100:00100000:3.0:1789303048.465683:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 715 00000100:00100000:3.0:1789303048.465691:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350627456:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303048.465726:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350627456:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (1620us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303048.465732:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 715 00000100:00100000:3.0:1789303048.465801:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 716 00000100:00100000:3.0:1789303048.465809:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350627712:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303048.465850:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d57480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350627712:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (1723us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303048.465856:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 716 00000100:00080000:0.0:1789303048.466321:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303048.466325:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789303048.466701:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303048.466761:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627840 00000100:00100000:0.0:1789303048.466766:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627840 00000100:00100000:0.0:1789303048.466811:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 717 00000100:00100000:0.0:1789303048.466818:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350627840:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303048.466869:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350627840:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 52us (170us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303048.466875:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 717 00000100:00080000:3.0:1789303048.467271:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000040:00080000:0.0:1789303048.593123:0:26758:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303048.593133:0:26758:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303048.594083:0:26758:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303048.594099:0:26758:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303048.851118:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303049.870863:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303050.444810:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303050.444820:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303050.445380:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303050.445400:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789303050.638149:0:26810:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303050.638159:0:26810:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303050.638760:0:26810:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303050.638769:0:26810:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303050.892856:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789303051.916699:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303052.303101:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d57100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371224832:0@lo:400:kworker.0 00000100:00100000:1.0:1789303052.303101:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d55180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371224576:0@lo:400:kworker.0 00000100:00100000:2.0:1789303052.303158:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d54a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371224448:0@lo:400:kworker.0 00000100:00100000:1.0:1789303052.303969:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303052.304148:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d56d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371225088:0@lo:400:kworker.0 00000100:00100000:1.0:1789303052.304184:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303052.304234:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d54e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371224704:0@lo:400:kworker.0 00000100:00100000:1.0:1789303052.304257:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303052.304294:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225088 00000100:00100000:1.0:1789303052.304318:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225088 00000100:00100000:1.0:1789303052.304585:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 718 00000100:00100000:1.0:1789303052.304598:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371225088:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303052.304671:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371225088:12345-0@lo:400:kworker.0 Request processed in 76us (485us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303052.304679:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 718 00000100:00100000:1.0:1789303052.304713:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224576 00000100:00100000:1.0:1789303052.304717:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224576 00000100:00100000:1.0:1789303052.304742:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224704 00000100:00100000:1.0:1789303052.304745:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224704 00000100:00100000:1.0:1789303052.304769:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:1.0:1789303052.304776:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371224576:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303052.304807:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371224576:12345-0@lo:400:kworker.0 Request processed in 32us (844us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303052.304814:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:0.0:1789303052.304819:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:0.0:1789303052.304828:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371224704:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789303052.304831:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789303052.304836:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789303052.304869:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371224704:12345-0@lo:400:kworker.0 Request processed in 43us (610us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303052.304874:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1789303052.304878:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d55180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371224576:0@lo:400:kworker.0 00000100:00100000:3.0:1789303052.305265:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303052.305330:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224832 00000100:00100000:3.0:1789303052.305335:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224832 00000100:00100000:3.0:1789303052.305374:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 719 00000100:00100000:3.0:1789303052.305381:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371224832:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303052.305421:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d56300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371224832:12345-0@lo:400:kworker.0 Request processed in 41us (157us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303052.305428:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 719 00000100:00100000:3.0:1789303052.305530:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d57100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371224832:0@lo:400:kworker.0 00000100:00100000:2.0:1789303052.306823:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303052.306883:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04d54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371224960:0@lo:400:kworker.0 00000100:00100000:3.0:1789303052.306905:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224448 00000100:00100000:3.0:1789303052.306914:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224448 00000100:00100000:2.0:1789303052.306926:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303052.306955:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:3.0:1789303052.306963:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371224448:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303052.306976:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371224960 00000100:00100000:2.0:1789303052.306979:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371224960 00000100:00100000:3.0:1789303052.307019:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371224448:12345-0@lo:400:kworker.0 Request processed in 57us (200us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303052.307027:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00100000:3.0:1789303052.307076:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d54a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371224448:0@lo:400:kworker.0 00000100:00100000:1.0:1789303052.307155:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d56d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371225088:0@lo:400:kworker.0 00000100:00080000:1.0:1789303052.307183:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789303052.307188:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789303052.307215:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d54e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371224704:0@lo:400:kworker.0 00000100:00100000:2.0:1789303052.307253:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:2.0:1789303052.307263:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ceb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371224960:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303052.307322:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ceb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371224960:12345-0@lo:400:kworker.0 Request processed in 60us (394us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303052.307349:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04d54700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371224960:0@lo:400:kworker.0 00000100:00100000:2.0:1789303052.307960:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000040:00080000:0.0:1789303052.640338:0:26861:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303052.640352:0:26861:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303052.641014:0:26861:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303052.641028:0:26861:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:2.0:1789303052.814855:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371225216:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303052.814951:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303052.822821:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225216 00000100:00100000:1.0:1789303052.822981:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225216 00000100:00100000:1.0:1789303052.823067:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1789303052.823079:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371225216:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303052.823135:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303052.823568:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371225216:12345-0@lo:13:osp-pre-1-0.0 Request processed in 490us (8614us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303052.823578:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:3.0:1789303052.827152:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371225216:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303052.941696:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789303053.325691:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371225344:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303053.325786:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303053.325941:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225344 00000100:00100000:1.0:1789303053.325950:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225344 00000100:00100000:1.0:1789303053.326008:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 283 00000100:00100000:1.0:1789303053.326018:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371225344:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789303053.326065:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303053.327351:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef40e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371225344:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1326us (1557us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303053.327364:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 283 00000100:00100000:0.0:1789303053.329941:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371225344:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303053.599049:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303053.599181:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303053.599227:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303053.599270:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303053.599342:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350627968 00000100:00100000:0.0:1789303053.599356:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350627968 00000100:00100000:0.0:1789303053.599433:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 686 00000100:00100000:0.0:1789303053.599444:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350627968:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303053.599581:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350627968:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 138us (399us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303053.599590:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 686 00000100:00100000:2.0:1789303053.601359:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628224 00000100:00100000:2.0:1789303053.601386:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628224 00000100:00100000:2.0:1789303053.601490:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 720 00000100:00100000:2.0:1789303053.601547:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350628224:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303053.601633:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350628224:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 134us (2405us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303053.601641:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 720 00000100:00080000:0.0:1789303053.603338:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789303053.603926:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303053.603954:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303053.603982:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303053.604008:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628736 00000100:00100000:3.0:1789303053.604020:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628736 00000100:00100000:3.0:1789303053.604079:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628352 00000100:00100000:3.0:1789303053.604082:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628352 00000100:00100000:3.0:1789303053.604101:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628608 00000100:00100000:3.0:1789303053.604104:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628608 00000100:00100000:3.0:1789303053.604126:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 721 00000100:00100000:3.0:1789303053.604138:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350628736:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303053.604230:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350628736:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 94us (310us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303053.604238:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 721 00000100:00100000:3.0:1789303053.604255:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 722 00000100:00100000:3.0:1789303053.604261:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350628352:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303053.604287:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d56d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350628352:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (332us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303053.604302:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 722 00000100:00100000:3.0:1789303053.604315:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 723 00000100:00100000:3.0:1789303053.604322:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350628608:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303053.604347:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d55180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350628608:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303053.604353:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 723 00000100:00100000:1.0:1789303053.606686:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628096 00000100:00100000:1.0:1789303053.606708:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628096 00000100:00100000:1.0:1789303053.606817:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628480 00000100:00100000:1.0:1789303053.606821:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628480 00000100:00100000:1.0:1789303053.607042:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 667 00000100:00100000:1.0:1789303053.607056:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350628096:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303053.607157:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d54700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350628096:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 104us (8124us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303053.607165:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 667 00000100:00100000:1.0:1789303053.607701:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00100000:1.0:1789303053.607713:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04d57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350628480:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303053.607757:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04d57100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350628480:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 48us (8486us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303053.607763:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 668 00000100:00080000:0.0:1789303053.608099:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789303053.608107:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303053.608111:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:3.0:1789303053.971073:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303054.589842:0:26912:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303054.589855:0:26912:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303054.591863:0:26912:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303054.591879:0:26912:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303054.988738:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303056.014192:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303056.358269:0:26963:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303056.358291:0:26963:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303056.358880:0:26963:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303056.358892:0:26963:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303057.036736:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789303057.421312:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371225856:0@lo:400:kworker.0 00000100:00100000:2.0:1789303057.421384:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303057.421450:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303057.421457:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371225728:0@lo:400:kworker.0 00000100:00100000:2.0:1789303057.421481:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303057.421539:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225728 00000100:00100000:2.0:1789303057.421548:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225728 00000100:00100000:1.0:1789303057.421554:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04cea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371225600:0@lo:400:kworker.0 00000100:00100000:3.0:1789303057.421602:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371225472:0@lo:400:kworker.0 00000100:00100000:2.0:1789303057.421603:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 669 00000100:00100000:2.0:1789303057.421611:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371225728:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303057.421634:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303057.421658:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303057.421664:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371225984:0@lo:400:kworker.0 00000100:00100000:2.0:1789303057.421665:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371225728:12345-0@lo:400:kworker.0 Request processed in 53us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303057.421673:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 669 00000100:00100000:2.0:1789303057.421690:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225600 00000100:00100000:3.0:1789303057.421692:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303057.421693:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371226112:0@lo:400:kworker.0 00000100:00100000:2.0:1789303057.421694:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225600 00000100:00100000:0.0:1789303057.421758:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225856 00000100:00100000:0.0:1789303057.421762:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225856 00000100:00100000:1.0:1789303057.421789:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303057.421795:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 724 00000100:00100000:0.0:1789303057.421804:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ceb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371225856:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303057.421841:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ceb480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371225856:12345-0@lo:400:kworker.0 Request processed in 39us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303057.421853:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 724 00000100:00100000:0.0:1789303057.421864:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371226112 00000100:00100000:0.0:1789303057.421866:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371226112 00000100:00100000:0.0:1789303057.421887:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 725 00000100:00100000:0.0:1789303057.421899:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371226112:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303057.421928:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371226112:12345-0@lo:400:kworker.0 Request processed in 30us (141us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303057.421933:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 725 00000100:00100000:0.0:1789303057.422330:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371226112:0@lo:400:kworker.0 00000100:00100000:2.0:1789303057.424537:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225984 00000100:00100000:1.0:1789303057.424574:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:1.0:1789303057.424603:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371225600:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303057.424689:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371225600:12345-0@lo:400:kworker.0 Request processed in 96us (3032us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303057.424697:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:1.0:1789303057.424761:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04cea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371225600:0@lo:400:kworker.0 00000100:00100000:1.0:1789303057.424821:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371225472 00000100:00100000:1.0:1789303057.424827:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225472 00000100:00100000:1.0:1789303057.424860:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 687 00000100:00100000:1.0:1789303057.424867:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371225472:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303057.424905:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371225472:12345-0@lo:400:kworker.0 Request processed in 38us (3269us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303057.424911:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 687 00000100:00080000:1.0:1789303057.425073:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789303057.425078:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789303057.425081:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303057.425104:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371225728:0@lo:400:kworker.0 00000100:00100000:1.0:1789303057.425116:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371225856:0@lo:400:kworker.0 00000100:00100000:1.0:1789303057.425310:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371225472:0@lo:400:kworker.0 00000100:00100000:2.0:1789303057.425811:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371225984 00000100:00100000:2.0:1789303057.425868:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:2.0:1789303057.425876:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371225984:12345-0@lo:400:kworker.0 00000100:00080000:1.0:1789303057.425987:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789303057.426073:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371225984:12345-0@lo:400:kworker.0 Request processed in 198us (4380us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303057.426081:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00000100:00100000:1.0:1789303057.426149:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371225984:0@lo:400:kworker.0 00000100:00100000:0.0:1789303057.932800:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371226240:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789303057.932904:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303057.933170:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371226240 00000100:00100000:3.0:1789303057.933181:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371226240 00000100:00100000:3.0:1789303057.933253:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1789303057.933273:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371226240:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789303057.933322:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789303057.933394:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef42a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371226240:12345-0@lo:13:osp-pre-1-0.0 Request processed in 132us (491us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303057.933402:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00100000:0.0:1789303057.933814:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371226240:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1789303058.060779:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303058.322029:0:27015:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303058.322044:0:27015:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303058.322716:0:27015:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303058.322730:0:27015:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789303058.458709:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303058.465641:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f044700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371226368:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303058.465715:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303058.465823:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371226368 00000100:00100000:1.0:1789303058.465832:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371226368 00000100:00100000:1.0:1789303058.465885:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 285 00000100:00100000:1.0:1789303058.465895:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371226368:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789303058.465929:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303058.465974:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371226368:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303058.465980:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 285 00000100:00100000:3.0:1789303058.466145:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f044700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371226368:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789303058.699876:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303058.699922:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303058.699956:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303058.699983:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303058.700005:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303058.700026:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303058.700053:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350629120 00000100:00100000:2.0:1789303058.700063:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350629120 00000100:00100000:2.0:1789303058.700116:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350629632 00000100:00100000:2.0:1789303058.700120:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350629632 00000100:00100000:2.0:1789303058.700135:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350629248 00000100:00100000:2.0:1789303058.700137:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350629248 00000100:00100000:2.0:1789303058.700158:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 726 00000100:00100000:2.0:1789303058.700168:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04cea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350629120:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303058.700236:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04cea680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350629120:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 69us (313us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303058.700244:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 726 00000100:00100000:2.0:1789303058.700351:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 727 00000100:00100000:2.0:1789303058.700359:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350629632:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303058.700384:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce9500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350629632:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303058.700391:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 727 00000100:00100000:2.0:1789303058.700402:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 728 00000100:00100000:2.0:1789303058.700408:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04cebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350629248:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303058.700427:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04cebb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350629248:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (401us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303058.700434:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 728 00000100:00080000:2.0:1789303058.700449:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303058.700453:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789303058.700551:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350629376 00000100:00100000:1.0:1789303058.700566:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350629376 00000100:00100000:1.0:1789303058.700616:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628992 00000100:00100000:1.0:1789303058.700619:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628992 00000100:00100000:1.0:1789303058.700646:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:1.0:1789303058.700655:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350629376:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303058.700689:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce9f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350629376:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (733us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303058.700697:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 672 00000100:00100000:1.0:1789303058.700720:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 673 00000100:00100000:3.0:1789303058.700731:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303058.700734:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ceb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350628992:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303058.700764:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ceb800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350628992:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (758us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303058.700770:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 673 00000100:00080000:1.0:1789303058.700808:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303058.700894:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350628864 00000100:00100000:0.0:1789303058.700900:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350628864 00000100:00100000:3.0:1789303058.700921:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350629504 00000100:00100000:3.0:1789303058.700932:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350629504 00000100:00100000:0.0:1789303058.700945:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 688 00000100:00100000:0.0:1789303058.700959:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350628864:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303058.701004:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 729 00000100:00100000:0.0:1789303058.701011:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bcc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350628864:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 51us (1139us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303058.701025:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350629504:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303058.701025:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 688 00000100:00100000:3.0:1789303058.701097:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350629504:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 77us (378us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303058.701105:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 729 00000100:00080000:1.0:1789303058.701197:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00010000:00010000:3.0:1789303059.084749:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303060.108717:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303060.398049:0:27066:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303060.398059:0:27066:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303060.399240:0:27066:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303060.399263:0:27066:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789303060.684692:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303060.684702:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303060.685242:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303060.685264:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789303061.133218:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303062.160708:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303062.547616:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371226624:0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.547689:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303062.547745:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03983800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371227136:0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.547771:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303062.547796:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789303062.547802:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371226496:0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.547830:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303062.547851:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371227008:0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.547871:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303062.548436:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371226880:0@lo:400:kworker.0 00000100:00100000:1.0:1789303062.548471:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303062.548495:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789303062.548520:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371226752:0@lo:400:kworker.0 00000100:00100000:1.0:1789303062.548542:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303062.549134:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227136 00000100:00100000:3.0:1789303062.549144:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227136 00000100:00100000:3.0:1789303062.549204:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371226880 00000100:00100000:3.0:1789303062.549207:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371226880 00000100:00100000:3.0:1789303062.549249:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 730 00000100:00100000:3.0:1789303062.549260:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371227136:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.549337:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371227136:12345-0@lo:400:kworker.0 Request processed in 79us (1565us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303062.549344:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 730 00000100:00100000:3.0:1789303062.549362:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 731 00000100:00100000:3.0:1789303062.549369:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371226880:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.549403:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371226880:12345-0@lo:400:kworker.0 Request processed in 34us (931us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303062.549408:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 731 00000100:00100000:3.0:1789303062.549446:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03983800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371227136:0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.549475:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371226880:0@lo:400:kworker.0 00000100:00100000:1.0:1789303062.550567:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371226496 00000100:00100000:1.0:1789303062.550577:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371226496 00000100:00100000:1.0:1789303062.550614:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 689 00000100:00100000:1.0:1789303062.550622:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371226496:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303062.550702:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bc850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371226496:12345-0@lo:400:kworker.0 Request processed in 81us (2870us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303062.550708:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 689 00000100:00100000:1.0:1789303062.550741:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371226624 00000100:00100000:1.0:1789303062.550745:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371226624 00000100:00100000:1.0:1789303062.550770:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227008 00000100:00100000:1.0:1789303062.550773:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227008 00000100:00100000:1.0:1789303062.550791:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371226752 00000100:00100000:1.0:1789303062.550794:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371226752 00000100:00100000:1.0:1789303062.550808:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:1.0:1789303062.550813:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371226624:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303062.550845:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371226624:12345-0@lo:400:kworker.0 Request processed in 31us (3156us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303062.550852:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:1.0:1789303062.550863:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:1.0:1789303062.550869:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371227008:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303062.551094:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:0.0:1789303062.551105:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371226752:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.551407:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371226496:0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.551423:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371227008:0@lo:400:kworker.0 00000100:00100000:3.0:1789303062.551434:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371226624:0@lo:400:kworker.0 00000100:00100000:1.0:1789303062.552074:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371227008:12345-0@lo:400:kworker.0 Request processed in 1202us (4199us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303062.552084:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00000100:00080000:1.0:1789303062.552129:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789303062.552134:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789303062.552136:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789303062.553474:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371226752:12345-0@lo:400:kworker.0 Request processed in 2364us (4923us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303062.553486:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:3.0:1789303062.553660:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371226752:0@lo:400:kworker.0 00000040:00080000:0.0:1789303062.577878:0:27117:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303062.579265:0:27117:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303062.579843:0:27117:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303062.579858:0:27117:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789303063.052791:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371227264:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789303063.052858:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303063.053417:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227264 00000100:00100000:2.0:1789303063.053425:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227264 00000100:00100000:2.0:1789303063.053486:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1789303063.053496:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371227264:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789303063.053554:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789303063.053613:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371227264:12345-0@lo:13:osp-pre-1-0.0 Request processed in 118us (754us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303063.053620:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:0.0:1789303063.053717:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371227264:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303063.180743:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789303063.564930:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789303063.564955:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f047800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371227392:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303063.565027:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303063.565160:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227392 00000100:00100000:3.0:1789303063.565170:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227392 00000100:00100000:3.0:1789303063.565218:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 287 00000100:00100000:3.0:1789303063.565236:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371227392:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1789303063.565271:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:3.0:1789303063.565326:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371227392:12345-0@lo:13:osp-pre-0-0.0 Request processed in 100us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303063.565333:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 287 00000100:00100000:0.0:1789303063.565426:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f047800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371227392:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303063.823940:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303063.824049:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303063.824080:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303063.824111:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303063.824132:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303063.824150:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303063.824201:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630144 00000100:00100000:2.0:1789303063.824210:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630144 00000100:00100000:2.0:1789303063.824277:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630400 00000100:00100000:2.0:1789303063.824281:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630400 00000100:00100000:2.0:1789303063.824299:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630016 00000100:00100000:2.0:1789303063.824301:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630016 00000100:00100000:2.0:1789303063.824322:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 732 00000100:00100000:2.0:1789303063.824331:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350630144:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303063.824424:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350630144:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 94us (374us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303063.824432:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 732 00000100:00100000:2.0:1789303063.824449:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 733 00000100:00100000:2.0:1789303063.824455:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350630400:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303063.824480:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350630400:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303063.824486:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 733 00000100:00100000:2.0:1789303063.824496:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 734 00000100:00100000:2.0:1789303063.824521:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350630016:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303063.824544:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350630016:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 42us (411us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303063.824550:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 734 00000100:00080000:2.0:1789303063.824903:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303063.824907:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789303063.824918:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350629760 00000100:00100000:2.0:1789303063.824922:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350629760 00000100:00100000:2.0:1789303063.824951:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 690 00000100:00100000:2.0:1789303063.825021:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350629760:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303063.825054:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350629760:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (942us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303063.825061:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 690 00000100:00100000:3.0:1789303063.828815:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303063.828945:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630528 00000100:00100000:3.0:1789303063.832204:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630528 00000100:00100000:3.0:1789303063.832760:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 735 00000100:00100000:3.0:1789303063.832774:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350630528:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303063.832852:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350630528:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 82us (4047us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303063.832860:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 735 00000100:00080000:3.0:1789303063.833253:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789303063.837032:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350629888 00000100:00100000:1.0:1789303063.837045:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350629888 00000100:00100000:1.0:1789303063.837116:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630272 00000100:00100000:1.0:1789303063.837118:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630272 00000100:00100000:1.0:1789303063.837360:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 677 00000100:00100000:1.0:1789303063.837371:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350629888:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303063.837459:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f047100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2346:x1876219350629888:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 90us (13528us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303063.837467:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 677 00000100:00100000:1.0:1789303063.837496:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 678 00000100:00100000:1.0:1789303063.837521:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350630272:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303063.837545:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f044e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350630272:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (13394us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303063.837551:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 678 00000100:00080000:3.0:1789303063.838251:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:0.0:1789303064.204751:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303064.673980:0:27171:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303064.673989:0:27171:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303064.682841:0:27171:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303064.682862:0:27171:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303065.228919:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789303066.254703:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303066.592432:0:27222:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303066.592442:0:27222:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303066.592995:0:27222:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303066.593007:0:27222:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:0.0:1789303067.276784:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303067.661940:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371227648:0@lo:400:kworker.0 00000100:00100000:3.0:1789303067.662005:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303067.662072:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371228160:0@lo:400:kworker.0 00000100:00100000:3.0:1789303067.662097:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303067.662117:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789303067.662122:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371227520:0@lo:400:kworker.0 00000100:00100000:3.0:1789303067.662146:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303067.662164:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371228032:0@lo:400:kworker.0 00000100:00100000:3.0:1789303067.662183:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303067.662216:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371227776:0@lo:400:kworker.0 00000100:00100000:3.0:1789303067.662238:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303067.662252:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303067.662258:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371227904:0@lo:400:kworker.0 00000100:00100000:3.0:1789303067.662294:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303067.662324:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227648 00000100:00100000:3.0:1789303067.662332:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227648 00000100:00100000:3.0:1789303067.662367:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228032 00000100:00100000:3.0:1789303067.662370:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228032 00000100:00100000:3.0:1789303067.662392:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227776 00000100:00100000:3.0:1789303067.662395:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227776 00000100:00100000:3.0:1789303067.662413:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 679 00000100:00100000:3.0:1789303067.662420:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371227648:12345-0@lo:400:kworker.0 00000100:00080000:0.0:1789303067.662424:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303067.662480:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371227648:12345-0@lo:400:kworker.0 Request processed in 59us (474us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303067.662487:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 679 00000100:00100000:3.0:1789303067.662524:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 680 00000100:00100000:3.0:1789303067.662531:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371228032:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303067.662562:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227520 00000100:00100000:3.0:1789303067.662563:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371228032:12345-0@lo:400:kworker.0 Request processed in 32us (378us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303067.662569:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227520 00000100:00100000:3.0:1789303067.662577:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 680 00000100:00100000:3.0:1789303067.662592:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 681 00000100:00100000:3.0:1789303067.662600:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371227776:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303067.662603:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 691 00000100:00100000:1.0:1789303067.662616:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371227520:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303067.662640:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371227776:12345-0@lo:400:kworker.0 Request processed in 40us (399us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303067.662647:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 681 00000100:00100000:1.0:1789303067.662660:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371227520:12345-0@lo:400:kworker.0 Request processed in 46us (511us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303067.662667:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 691 00000100:00100000:3.0:1789303067.662688:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371227776:0@lo:400:kworker.0 00000100:00100000:0.0:1789303067.662836:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f045f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371227520:0@lo:400:kworker.0 00000100:00100000:0.0:1789303067.662846:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371228032:0@lo:400:kworker.0 00000100:00100000:0.0:1789303067.662854:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f046d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371227648:0@lo:400:kworker.0 00000100:00100000:2.0:1789303067.663460:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228160 00000100:00100000:2.0:1789303067.663467:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228160 00000100:00100000:2.0:1789303067.663553:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371227904 00000100:00100000:2.0:1789303067.663558:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371227904 00000100:00100000:1.0:1789303067.663562:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 736 00000100:00100000:1.0:1789303067.663573:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371228160:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303067.663588:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 737 00000100:00100000:2.0:1789303067.663598:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371227904:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303067.663625:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371228160:12345-0@lo:400:kworker.0 Request processed in 55us (1527us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303067.663633:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 736 00000100:00100000:2.0:1789303067.663652:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371227904:12345-0@lo:400:kworker.0 Request processed in 55us (1356us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303067.663660:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 737 00000100:00100000:1.0:1789303067.663666:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371228160:0@lo:400:kworker.0 00000100:00100000:1.0:1789303067.663719:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371227904:0@lo:400:kworker.0 00000100:00080000:0.0:1789303067.663809:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303067.663813:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789303068.172692:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303068.173006:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371228288:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303068.173072:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303068.173204:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228288 00000100:00100000:3.0:1789303068.173212:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228288 00000100:00100000:3.0:1789303068.173313:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:3.0:1789303068.173323:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371228288:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789303068.173361:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789303068.173416:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371228288:12345-0@lo:13:osp-pre-1-0.0 Request processed in 93us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303068.173423:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:2.0:1789303068.173930:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03982680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371228288:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303068.301130:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303068.507910:0:27273:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303068.507919:0:27273:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303068.508917:0:27273:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303068.508931:0:27273:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789303068.685497:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371228416:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303068.685585:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303068.685946:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228416 00000100:00100000:2.0:1789303068.685954:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228416 00000100:00100000:2.0:1789303068.686006:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 289 00000100:00100000:2.0:1789303068.686015:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371228416:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303068.686048:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303068.686142:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371228416:12345-0@lo:13:osp-pre-0-0.0 Request processed in 127us (557us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303068.686153:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 289 00000100:00100000:1.0:1789303068.687805:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371228416:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303068.945120:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303068.945167:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303068.945211:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303068.945599:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630784 00000100:00100000:0.0:1789303068.945609:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630784 00000100:00100000:0.0:1789303068.945678:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 682 00000100:00100000:0.0:1789303068.945688:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350630784:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303068.945762:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2345:x1876219350630784:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 76us (653us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303068.945773:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 682 00000100:00100000:1.0:1789303068.945857:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631040 00000100:00100000:1.0:1789303068.945862:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631040 00000100:00100000:1.0:1789303068.945900:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631296 00000100:00100000:1.0:1789303068.945903:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631296 00000100:00100000:1.0:1789303068.945923:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 738 00000100:00100000:1.0:1789303068.945930:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350631040:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303068.945959:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267dc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350631040:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (792us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303068.945966:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 738 00000100:00100000:1.0:1789303068.945978:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 739 00000100:00100000:1.0:1789303068.945984:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350631296:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303068.946006:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350631296:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (795us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303068.946012:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 739 00000100:00080000:3.0:1789303068.947561:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789303068.947570:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789303068.947573:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303068.947667:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303068.947899:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630656 00000100:00100000:1.0:1789303068.947909:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630656 00000100:00100000:1.0:1789303068.947960:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:1.0:1789303068.947969:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350630656:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303068.948033:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350630656:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 65us (370us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303068.948041:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:3.0:1789303068.949696:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303068.949771:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350630912 00000100:00100000:2.0:1789303068.949781:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350630912 00000100:00100000:3.0:1789303068.949793:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303068.949833:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303068.949884:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631168 00000100:00100000:1.0:1789303068.949889:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631168 00000100:00100000:1.0:1789303068.949925:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 683 00000100:00100000:1.0:1789303068.949934:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350631168:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303068.949995:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350631168:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 63us (204us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303068.950001:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 683 00000100:00100000:2.0:1789303068.950062:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631424 00000100:00100000:2.0:1789303068.950067:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631424 00000100:00100000:2.0:1789303068.950098:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 740 00000100:00100000:2.0:1789303068.950105:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350630912:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303068.950148:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 741 00000100:00100000:3.0:1789303068.950159:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350631424:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:0.0:1789303068.950169:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789303068.950175:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789303068.950178:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789303068.954231:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267d180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350630912:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 4114us (4530us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303068.954300:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 740 00000100:00100000:3.0:1789303068.956695:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267c700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350631424:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 6526us (6849us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303068.956711:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 741 00010000:00010000:2.0:1789303069.325601:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:0.0:1789303070.317960:0:27324:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303070.317970:0:27324:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303070.318542:0:27324:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303070.318554:0:27324:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303070.354952:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303070.924856:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303070.924866:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303070.925405:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303070.925416:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789303070.929171:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303070.929296:0:8463:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631552 00000100:00100000:1.0:1789303070.929310:0:8463:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631552 00000100:00100000:1.0:1789303070.929422:0:8463:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 81 00000100:00100000:1.0:1789303070.929435:0:8463:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+108:2347:x1876219350631552:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:1.0:1789303070.929456:0:8463:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 16 locks, starting at 0 00000100:00080000:0.0:1789303070.930920:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:1.0:1789303070.932155:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72200/0x64fe43095184ace6 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f709 expref: 108 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932194:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x10:0x0].0x0 (ffff960f02d72500) 00010000:00010000:1.0:1789303070.932231:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dce00/0x64fe43095184aa9a lrc: 0/0,0 mode: --/PR res: [0x200000401:0xf:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f661 expref: 107 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932242:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff960f421e9300) 00010000:00010000:1.0:1789303070.932250:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dd600/0x64fe43095184aa0e lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f63e expref: 106 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932261:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c800/0x64fe43095184a8fd lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x51/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f5f1 expref: 105 pid: 5844 timeout: 448 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932270:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff960f02d72a00) 00010000:00010000:1.0:1789303070.932277:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dca00/0x64fe43095184a7c2 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f58f expref: 104 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932291:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc:0x0].0x0 (ffff960f11d36a00) 00010000:00010000:1.0:1789303070.932299:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dda00/0x64fe43095184a72f lrc: 0/0,0 mode: --/PR res: [0x200000401:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f56c expref: 103 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932311:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7e600/0x64fe43095184a5ed lrc: 0/0,0 mode: --/CR res: [0x200000401:0xb:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f52d expref: 102 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932320:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xb:0x0].0x0 (ffff960f09ae1d00) 00010000:00010000:1.0:1789303070.932327:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef6200/0x64fe43095184a5bc lrc: 0/0,0 mode: --/PR res: [0x200000401:0xa:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f51f expref: 101 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932335:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xa:0x0].0x0 (ffff960f02d72f00) 00010000:00010000:1.0:1789303070.932345:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc600/0x64fe43095184a553 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f50a expref: 100 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932353:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x8:0x0].0x0 (ffff960f11d36100) 00010000:00010000:1.0:1789303070.932368:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662400/0x64fe43095184a315 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f462 expref: 99 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932377:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff960f02d72c00) 00010000:00010000:1.0:1789303070.932403:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dd000/0x64fe43095184a1b0 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f3f9 expref: 98 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932413:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0x2:0x0].0x0 (ffff960f11d36500) 00010000:00010000:1.0:1789303070.932423:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dc400/0x64fe43095184a0bb lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f3a5 expref: 97 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932432:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff960f11d36000) 00010000:00010000:1.0:1789303070.932440:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f73200/0x64fe430951849f87 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f351 expref: 96 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932452:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f01662600/0x64fe430951849eed lrc: 0/0,0 mode: --/PR res: [0x200000401:0x4:0x0].0x0 bits 0x1a/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f335 expref: 95 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932464:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f000/0x64fe430951849dff lrc: 0/0,0 mode: --/CR res: [0x200000401:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f304 expref: 94 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932473:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x4:0x0].0x0 (ffff960f09ae1300) 00010000:00010000:1.0:1789303070.932482:0:8463:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f72e00/0x64fe430951849dce lrc: 0/0,0 mode: --/PR res: [0x200000401:0x3:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f2f6 expref: 93 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:1.0:1789303070.932490:0:8463:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x3:0x0].0x0 (ffff960f421e9f00) 00010000:00010000:1.0:1789303070.932496:0:8463:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1789303070.932621:0:8463:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2347:x1876219350631552:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 3187us (3480us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303070.932631:0:8463:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 81 00010000:00010000:3.0:1789303071.379949:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303072.292567:0:27376:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303072.292577:0:27376:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303072.293084:0:27376:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303072.293094:0:27376:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303072.401821:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303072.780889:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371228672:0@lo:400:kworker.0 00000100:00100000:1.0:1789303072.780960:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303072.781018:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371229184:0@lo:400:kworker.0 00000100:00100000:1.0:1789303072.781044:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303072.781080:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789303072.781087:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371228544:0@lo:400:kworker.0 00000100:00100000:1.0:1789303072.781114:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303072.781136:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371229056:0@lo:400:kworker.0 00000100:00100000:1.0:1789303072.781155:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303072.781190:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371228800:0@lo:400:kworker.0 00000100:00100000:1.0:1789303072.781212:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303072.781229:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789303072.781234:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371228928:0@lo:400:kworker.0 00000100:00100000:1.0:1789303072.781253:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303072.781282:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228672 00000100:00100000:1.0:1789303072.781289:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228672 00000100:00100000:1.0:1789303072.781330:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229056 00000100:00100000:1.0:1789303072.781333:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229056 00000100:00100000:1.0:1789303072.781360:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228800 00000100:00100000:1.0:1789303072.781363:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228800 00000100:00100000:1.0:1789303072.781385:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:1.0:1789303072.781394:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371228672:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303072.781399:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229184 00000100:00100000:2.0:1789303072.781406:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229184 00000100:00100000:2.0:1789303072.781440:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228928 00000100:00100000:2.0:1789303072.781443:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228928 00000100:00100000:2.0:1789303072.781480:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 742 00000100:00100000:2.0:1789303072.781488:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371229184:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303072.781557:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371229184:12345-0@lo:400:kworker.0 Request processed in 69us (510us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303072.781565:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 742 00000100:00100000:2.0:1789303072.781583:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 743 00000100:00100000:2.0:1789303072.781589:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371228928:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303072.781683:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:0.0:1789303072.781693:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371229056:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303072.781734:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371229056:12345-0@lo:400:kworker.0 Request processed in 42us (577us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303072.781740:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00100000:0.0:1789303072.781755:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:0.0:1789303072.781761:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371228800:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303072.781832:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371228544 00000100:00100000:3.0:1789303072.781837:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371228544 00000100:00100000:2.0:1789303072.783382:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371228928:12345-0@lo:400:kworker.0 Request processed in 1786us (2120us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303072.783395:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 743 00000100:00080000:2.0:1789303072.784371:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303072.784379:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303072.784383:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303072.787243:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371228800:12345-0@lo:400:kworker.0 Request processed in 5472us (6019us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303072.787258:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00100000:2.0:1789303072.787498:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371228928:0@lo:400:kworker.0 00000100:00100000:2.0:1789303072.787536:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371228800:0@lo:400:kworker.0 00000100:00100000:2.0:1789303072.787582:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371229056:0@lo:400:kworker.0 00000100:00100000:2.0:1789303072.787594:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371229184:0@lo:400:kworker.0 00000100:00100000:1.0:1789303072.787759:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53800 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371228672:12345-0@lo:400:kworker.0 Request processed in 6364us (6797us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303072.787768:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00100000:2.0:1789303072.787890:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371228672:0@lo:400:kworker.0 00000100:00100000:3.0:1789303072.788075:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 693 00000100:00100000:3.0:1789303072.788087:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371228544:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303072.788146:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bf450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371228544:12345-0@lo:400:kworker.0 Request processed in 61us (7030us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303072.788153:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 693 00000100:00100000:0.0:1789303072.789201:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371228544:0@lo:400:kworker.0 00000100:00100000:2.0:1789303073.292844:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371229312:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303073.292919:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303073.293256:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229312 00000100:00100000:3.0:1789303073.293265:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229312 00000100:00100000:3.0:1789303073.293313:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:3.0:1789303073.293321:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371229312:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789303073.293353:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789303073.293409:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce9c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371229312:12345-0@lo:13:osp-pre-1-0.0 Request processed in 89us (490us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303073.293415:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:0.0:1789303073.293553:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03981880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371229312:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303073.420929:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303073.805710:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371229440:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303073.805792:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303073.805903:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229440 00000100:00100000:2.0:1789303073.805911:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229440 00000100:00100000:2.0:1789303073.805964:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 291 00000100:00100000:2.0:1789303073.805974:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371229440:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303073.806006:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303073.806053:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371229440:12345-0@lo:13:osp-pre-0-0.0 Request processed in 81us (261us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303073.806059:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 291 00000100:00100000:2.0:1789303073.806103:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371229440:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303074.064367:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303074.064408:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303074.064433:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303074.064451:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303074.064467:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303074.064483:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303074.064523:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303074.064552:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631680 00000100:00100000:1.0:1789303074.064561:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631680 00000100:00100000:1.0:1789303074.064605:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 694 00000100:00100000:1.0:1789303074.064614:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350631680:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303074.064678:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bfc50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2345:x1876219350631680:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 65us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303074.064686:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 694 00000100:00100000:1.0:1789303074.064716:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631936 00000100:00100000:1.0:1789303074.064720:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631936 00000100:00100000:1.0:1789303074.064752:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632448 00000100:00100000:1.0:1789303074.064755:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632448 00000100:00100000:1.0:1789303074.064769:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632064 00000100:00100000:1.0:1789303074.064771:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632064 00000100:00100000:1.0:1789303074.064781:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632320 00000100:00100000:1.0:1789303074.064782:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632320 00000100:00100000:1.0:1789303074.064794:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 744 00000100:00100000:1.0:1789303074.064799:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350631936:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303074.064824:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350631936:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (416us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303074.064830:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 744 00000100:00100000:1.0:1789303074.064840:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 745 00000100:00100000:1.0:1789303074.064845:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350632448:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303074.064864:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2345:x1876219350632448:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 19us (413us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303074.064869:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 745 00000100:00100000:1.0:1789303074.064877:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 746 00000100:00100000:1.0:1789303074.064882:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350632064:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303074.064900:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2345:x1876219350632064:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 17us (415us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303074.064905:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 746 00000100:00100000:1.0:1789303074.064916:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 747 00000100:00100000:1.0:1789303074.064922:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350632320:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303074.064942:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2345:x1876219350632320:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303074.064948:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 747 00000100:00100000:1.0:1789303074.064967:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632192 00000100:00100000:1.0:1789303074.064971:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632192 00000100:00100000:1.0:1789303074.064995:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350631808 00000100:00100000:1.0:1789303074.064997:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350631808 00000100:00100000:1.0:1789303074.065017:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 687 00000100:00100000:1.0:1789303074.065024:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b86680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350632192:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303074.065050:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b86680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2345:x1876219350632192:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 27us (617us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303074.065056:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 687 00000100:00100000:1.0:1789303074.065068:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 688 00000100:00100000:1.0:1789303074.065075:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b86300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2345:x1876219350631808:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303074.065098:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b86300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2345:x1876219350631808:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (630us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303074.065104:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 688 00000100:00080000:1.0:1789303074.065263:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789303074.065268:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789303074.065270:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000040:00080000:0.0:1789303074.204795:0:27427:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303074.204804:0:27427:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303074.205935:0:27427:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303074.205953:0:27427:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303074.444713:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789303075.468725:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303076.109888:0:27478:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303076.109898:0:27478:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303076.116318:0:27478:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303076.116342:0:27478:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303076.492762:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303077.521340:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303077.901722:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b87100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371229952:0@lo:400:kworker.0 00000100:00100000:1.0:1789303077.901802:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303077.901857:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789303077.901863:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371229824:0@lo:400:kworker.0 00000100:00100000:1.0:1789303077.901887:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303077.901919:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229952 00000100:00100000:1.0:1789303077.901926:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229952 00000100:00100000:1.0:1789303077.901978:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 748 00000100:00100000:1.0:1789303077.901985:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371229952:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303077.902032:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b85880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371229952:12345-0@lo:400:kworker.0 Request processed in 47us (230us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303077.902039:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 748 00000100:00100000:1.0:1789303077.902064:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229824 00000100:00100000:1.0:1789303077.902067:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229824 00000100:00100000:1.0:1789303077.902090:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 689 00000100:00100000:1.0:1789303077.902096:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371229824:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303077.902121:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371229824:12345-0@lo:400:kworker.0 Request processed in 26us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303077.902136:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 689 00000100:00080000:1.0:1789303077.902151:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789303077.902155:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:1.0:1789303077.902186:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371229824:0@lo:400:kworker.0 00000100:00100000:1.0:1789303077.902196:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b87100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371229952:0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.906727:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b86a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371229568:0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.906877:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303077.906924:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371230080:0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.906952:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303077.906988:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1789303077.906996:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371229696:0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.907017:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303077.907035:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371230208:0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.907054:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303077.907091:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230080 00000100:00100000:0.0:1789303077.907098:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230080 00000100:00100000:0.0:1789303077.907157:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229696 00000100:00100000:0.0:1789303077.907161:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229696 00000100:00100000:0.0:1789303077.907181:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 690 00000100:00100000:0.0:1789303077.907190:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371230080:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.907251:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03981880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371230080:12345-0@lo:400:kworker.0 Request processed in 62us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303077.907258:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 690 00000100:00100000:0.0:1789303077.907276:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 691 00000100:00100000:0.0:1789303077.907283:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371229696:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.907306:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3302:x1876219371229696:12345-0@lo:400:kworker.0 Request processed in 24us (289us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303077.907312:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 691 00000100:00100000:0.0:1789303077.907329:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230208 00000100:00100000:0.0:1789303077.907332:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230208 00000100:00100000:0.0:1789303077.907359:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 749 00000100:00100000:0.0:1789303077.907365:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371230208:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.907391:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371230208:12345-0@lo:400:kworker.0 Request processed in 27us (336us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303077.907397:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 749 00000100:00080000:0.0:1789303077.907409:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789303077.907412:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303077.907439:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b84a80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3302:1876219371229696:0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.907449:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b84e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371230208:0@lo:400:kworker.0 00000100:00100000:0.0:1789303077.907457:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371230080:0@lo:400:kworker.0 00000100:00100000:3.0:1789303077.907571:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371229568 00000100:00100000:3.0:1789303077.907575:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371229568 00000100:00100000:3.0:1789303077.907610:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 695 00000100:00100000:3.0:1789303077.907617:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371229568:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303077.907652:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334bd450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371229568:12345-0@lo:400:kworker.0 Request processed in 36us (775us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303077.907658:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 695 00000100:00100000:0.0:1789303077.907980:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b86a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371229568:0@lo:400:kworker.0 00000040:00080000:1.0:1789303078.089977:0:27529:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303078.089987:0:27529:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303078.091014:0:27529:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303078.091029:0:27529:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789303078.412913:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789303078.420009:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371230336:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1789303078.420101:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303078.420389:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230336 00000100:00100000:2.0:1789303078.420399:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230336 00000100:00100000:2.0:1789303078.420449:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:2.0:1789303078.420459:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371230336:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789303078.420496:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789303078.420573:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371230336:12345-0@lo:13:osp-pre-1-0.0 Request processed in 116us (471us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303078.420580:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:1.0:1789303078.420748:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e03980700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371230336:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789303078.540937:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303078.924790:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303078.924816:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371230464:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303078.924888:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303078.925876:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230464 00000100:00100000:2.0:1789303078.925885:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230464 00000100:00100000:2.0:1789303078.925937:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 293 00000100:00100000:2.0:1789303078.925948:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371230464:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303078.925994:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303078.926075:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267fb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371230464:12345-0@lo:13:osp-pre-0-0.0 Request processed in 129us (1187us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303078.926081:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 293 00000100:00100000:0.0:1789303078.927435:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371230464:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303079.180485:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303079.180576:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303079.180609:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303079.180638:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303079.180655:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303079.180673:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303079.180693:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303079.180720:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633088 00000100:00100000:2.0:1789303079.180723:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632832 00000100:00100000:0.0:1789303079.180729:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633088 00000100:00100000:2.0:1789303079.180730:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632832 00000100:00100000:1.0:1789303079.180808:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632576 00000100:00100000:1.0:1789303079.180813:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632576 00000100:00100000:2.0:1789303079.180846:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633344 00000100:00100000:2.0:1789303079.180852:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633344 00000100:00100000:1.0:1789303079.180857:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 696 00000100:00100000:1.0:1789303079.180868:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350632576:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303079.180875:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632960 00000100:00100000:2.0:1789303079.180879:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632960 00000100:00100000:2.0:1789303079.180894:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633216 00000100:00100000:2.0:1789303079.180897:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633216 00000100:00100000:2.0:1789303079.180915:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 750 00000100:00100000:2.0:1789303079.180923:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350632832:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303079.180938:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350632576:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 70us (461us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303079.180947:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 696 00000100:00100000:2.0:1789303079.180957:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350632832:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 34us (379us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303079.180964:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 750 00000100:00100000:2.0:1789303079.180978:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 751 00000100:00100000:2.0:1789303079.180984:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350633344:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303079.181004:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 752 00000100:00100000:2.0:1789303079.181011:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03982680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350633344:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (371us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303079.181018:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350632960:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303079.181024:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 751 00000100:00100000:2.0:1789303079.181039:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 753 00000100:00100000:2.0:1789303079.181047:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350633216:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303079.181056:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350632960:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (381us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303079.181064:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 752 00000100:00100000:2.0:1789303079.181078:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f046d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350633216:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (381us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303079.181086:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 753 00000100:00080000:2.0:1789303079.181202:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303079.181206:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303079.181209:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303079.182183:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350632704 00000100:00100000:3.0:1789303079.182203:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350632704 00000100:00100000:3.0:1789303079.182282:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:3.0:1789303079.182291:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2346:x1876219350632704:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303079.182340:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980a80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2346:x1876219350632704:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 50us (1683us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303079.182346:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 692 00000100:00100000:0.0:1789303079.182947:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 693 00000100:00100000:0.0:1789303079.182958:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350633088:12345-192.168.203.11@tcp:400:kworker.0 00000100:00080000:2.0:1789303079.183079:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303079.183274:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03980700 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2346:x1876219350633088:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 318us (2662us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303079.183284:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 693 00010000:00010000:3.0:1789303079.565405:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303080.032036:0:27581:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303080.032046:0:27581:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303080.032606:0:27581:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303080.032620:0:27581:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:1.0:1789303080.140705:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303080.140713:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303080.141435:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303080.141447:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303080.592753:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303081.165117:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303081.165303:0:13636:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633472 00000100:00100000:0.0:1789303081.165313:0:13636:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633472 00000100:00100000:0.0:1789303081.165397:0:13636:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 82 00000100:00100000:0.0:1789303081.165408:0:13636:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1bd06450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+52:2348:x1876219350633472:12345-192.168.203.11@tcp:103:ldlm_bl.0 00010000:00010000:0.0:1789303081.165433:0:13636:0:(ldlm_lockd.c:1747:ldlm_request_cancel()) ### server-side cancel handler START: 16 locks, starting at 0 00010000:00010000:0.0:1789303081.165518:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2c600/0x64fe43095184bf31 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x22:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fc34 expref: 52 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165540:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x22:0x0].0x0 (ffff960f14168500) 00010000:00010000:0.0:1789303081.165560:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98800/0x64fe43095184bda2 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fbaf expref: 51 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165572:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xd:0x0].0x0 (ffff960f421e9b00) 00010000:00010000:0.0:1789303081.165580:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dd200/0x64fe43095184bd32 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x21:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb8c expref: 50 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165588:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x21:0x0].0x0 (ffff960f421e9200) 00010000:00010000:0.0:1789303081.165595:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98200/0x64fe43095184bbd4 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fb15 expref: 49 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165603:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xc:0x0].0x0 (ffff960f421e9a00) 00010000:00010000:0.0:1789303081.165610:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3f6fbc00/0x64fe43095184bb64 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x20:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739faf2 expref: 48 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165620:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x20:0x0].0x0 (ffff960f02dfdc00) 00010000:00010000:0.0:1789303081.165628:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f73800/0x64fe43095184ba3e lrc: 0/0,0 mode: --/PR res: [0x200000402:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa90 expref: 47 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165636:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xb:0x0].0x0 (ffff960f14168b00) 00010000:00010000:0.0:1789303081.165648:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f73e00/0x64fe43095184b9f8 lrc: 0/0,0 mode: --/PR res: [0x200000402:0xa:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739fa7b expref: 46 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165659:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000402:0xa:0x0].0x0 (ffff960f14168e00) 00010000:00010000:0.0:1789303081.165667:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7f200/0x64fe43095184b2dc lrc: 0/0,0 mode: --/PR res: [0x200000401:0x16:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f891 expref: 45 pid: 6919 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165675:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x16:0x0].0x0 (ffff960f09ae1a00) 00010000:00010000:0.0:1789303081.165683:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dc200/0x64fe43095184b035 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x13:0x0].0x0 bits 0x1/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f7cd expref: 44 pid: 5845 timeout: 472 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165692:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x13:0x0].0x0 (ffff960f09ae1700) 00010000:00010000:0.0:1789303081.165699:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f121dcc00/0x64fe43095184b058 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x12:0x0].0x0 bits 0x11/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f7d4 expref: 43 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165707:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x12:0x0].0x0 (ffff960f02d72b00) 00010000:00010000:0.0:1789303081.165715:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f3fc98a00/0x64fe43095184ac14 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xf:0x0].0x0 bits 0x11/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f6ca expref: 42 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165725:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xf:0x0].0x0 (ffff960f421e9300) 00010000:00010000:0.0:1789303081.165754:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f25f73600/0x64fe43095184a935 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xd:0x0].0x0 bits 0x51/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f5f8 expref: 41 pid: 6919 timeout: 448 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165769:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xd:0x0].0x0 (ffff960f02d72a00) 00010000:00010000:0.0:1789303081.165782:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960e03c2d200/0x64fe43095184a958 lrc: 0/0,0 mode: --/PR res: [0x200000401:0xc:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f5ff expref: 40 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165792:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0xc:0x0].0x0 (ffff960f11d36a00) 00010000:00010000:0.0:1789303081.165801:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f07ef7600/0x64fe43095184a488 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x7:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f4cb expref: 39 pid: 5844 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165809:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x7:0x0].0x0 (ffff960f02d72c00) 00010000:00010000:0.0:1789303081.165821:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f10e7ea00/0x64fe43095184a07c lrc: 0/0,0 mode: --/PR res: [0x200000002:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f397 expref: 38 pid: 5843 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165830:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000002:0x1:0x0].0x0 (ffff960f09ae1400) 00010000:00010000:0.0:1789303081.165840:0:13636:0:(ldlm_lock.c:193:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: ffff960f120dce00/0x64fe43095184a052 lrc: 0/0,0 mode: --/PR res: [0x200000401:0x5:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.203.11@tcp remote: 0xed5c2244f739f389 expref: 37 pid: 5845 timeout: 0 lvb_type: 0 lru_score: 0 lru_type: 0 00010000:00010000:0.0:1789303081.165847:0:13636:0:(ldlm_inodebits.c:74:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000401:0x5:0x0].0x0 (ffff960f11d36000) 00010000:00010000:0.0:1789303081.165853:0:13636:0:(ldlm_lockd.c:1808:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1789303081.165914:0:13636:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1bd06450 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350633472:12345-192.168.203.11@tcp:103:ldlm_bl.0 Request processed in 508us (817us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303081.165922:0:13636:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 82 00000100:00080000:2.0:1789303081.166010:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:2.0:1789303081.613299:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303082.195863:0:27632:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303082.195877:0:27632:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303082.196424:0:27632:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303082.196436:0:27632:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303082.637492:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303083.020783:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1789303083.027110:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04cea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371230720:0@lo:400:kworker.0 00000100:00100000:1.0:1789303083.027185:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303083.027262:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371230592:0@lo:400:kworker.0 00000100:00100000:1.0:1789303083.027291:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303083.027431:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230592 00000100:00100000:0.0:1789303083.027442:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230592 00000100:00100000:0.0:1789303083.027495:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 697 00000100:00100000:0.0:1789303083.027526:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371230592:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303083.027594:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371230592:12345-0@lo:400:kworker.0 Request processed in 91us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303083.027602:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 697 00000100:00100000:1.0:1789303083.027612:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371230592:0@lo:400:kworker.0 00000100:00100000:3.0:1789303083.028708:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230720 00000100:00100000:3.0:1789303083.028716:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230720 00000100:00100000:3.0:1789303083.028765:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:3.0:1789303083.028774:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371230720:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303083.028851:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371230720:12345-0@lo:400:kworker.0 Request processed in 78us (1665us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303083.028858:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:0.0:1789303083.028915:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04cea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371230720:0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.029227:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04cea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371230976:0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.029266:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303083.029291:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303083.029297:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371230848:0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.029318:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303083.029346:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230976 00000100:00100000:2.0:1789303083.029350:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230976 00000100:00100000:2.0:1789303083.029388:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 754 00000100:00100000:2.0:1789303083.029395:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371230976:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.029442:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce9880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371230976:12345-0@lo:400:kworker.0 Request processed in 47us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303083.029449:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 754 00000100:00100000:2.0:1789303083.029469:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371230848 00000100:00100000:2.0:1789303083.029472:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371230848 00000100:00100000:2.0:1789303083.029497:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 695 00000100:00100000:2.0:1789303083.029520:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371230848:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.029549:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371230848:12345-0@lo:400:kworker.0 Request processed in 46us (229us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303083.029555:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 695 00000100:00080000:2.0:1789303083.029571:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:2.0:1789303083.029575:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:2.0:1789303083.029598:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371230848:0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.029610:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04cea300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371230976:0@lo:400:kworker.0 00000100:00100000:0.0:1789303083.029626:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371231232:0@lo:400:kworker.0 00000100:00100000:0.0:1789303083.029709:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303083.029745:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1789303083.029751:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371231104:0@lo:400:kworker.0 00000100:00100000:0.0:1789303083.029776:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303083.029846:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371231104 00000100:00100000:1.0:1789303083.029852:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371231104 00000100:00100000:2.0:1789303083.029860:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371231232 00000100:00100000:2.0:1789303083.029867:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371231232 00000100:00100000:1.0:1789303083.029902:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:2.0:1789303083.029906:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 755 00000100:00100000:1.0:1789303083.029913:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371231104:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.029919:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371231232:12345-0@lo:400:kworker.0 00000100:00080000:3.0:1789303083.030069:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789303083.030073:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303083.043563:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e03983100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371231104:12345-0@lo:400:kworker.0 Request processed in 13644us (13776us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303083.043579:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00100000:3.0:1789303083.044461:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f045f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371231232:12345-0@lo:400:kworker.0 Request processed in 14543us (14748us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303083.044473:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 755 00000100:00100000:0.0:1789303083.048418:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371231104:0@lo:400:kworker.0 00000100:00100000:0.0:1789303083.048447:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371231232:0@lo:400:kworker.0 00000100:00100000:2.0:1789303083.536233:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303083.536256:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371231360:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303083.536323:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303083.536427:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371231360 00000100:00100000:1.0:1789303083.536437:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371231360 00000100:00100000:1.0:1789303083.536494:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:1.0:1789303083.536532:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371231360:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303083.536742:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303083.536805:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371231360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 301us (481us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303083.536812:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00100000:3.0:1789303083.536888:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fb8e00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371231360:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303083.662181:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789303084.045349:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371231488:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303084.045418:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303084.045534:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371231488 00000100:00100000:1.0:1789303084.045543:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371231488 00000100:00100000:1.0:1789303084.045607:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 295 00000100:00100000:1.0:1789303084.045619:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371231488:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789303084.045653:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303084.057855:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6afb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371231488:12345-0@lo:13:osp-pre-0-0.0 Request processed in 12229us (12428us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303084.057870:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 295 00000100:00100000:0.0:1789303084.059021:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371231488:0@lo:13:osp-pre-0-0.0 00000040:00080000:0.0:1789303084.285887:0:27683:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:0.0:1789303084.285897:0:27683:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:0.0:1789303084.286759:0:27683:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:0.0:1789303084.286772:0:27683:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789303084.310148:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303084.310196:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303084.310225:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303084.310276:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633984 00000100:00100000:1.0:1789303084.310285:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633984 00000100:00100000:1.0:1789303084.310340:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350634240 00000100:00100000:1.0:1789303084.310344:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350634240 00000100:00100000:1.0:1789303084.310369:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 756 00000100:00100000:1.0:1789303084.310379:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350633984:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303084.310673:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fba680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350633984:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 295us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303084.310682:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 756 00000100:00100000:1.0:1789303084.310766:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 757 00000100:00100000:1.0:1789303084.310774:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350634240:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303084.310800:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350634240:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (573us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303084.310806:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 757 00000100:00100000:1.0:1789303084.310855:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303084.310912:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303084.311002:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303084.311026:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303084.311044:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633600 00000100:00100000:1.0:1789303084.311048:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633600 00000100:00100000:1.0:1789303084.311074:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:1.0:1789303084.311091:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350633600:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303084.311123:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a0850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350633600:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303084.311130:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:1.0:1789303084.311156:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633728 00000100:00100000:1.0:1789303084.311160:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633728 00000100:00100000:1.0:1789303084.311185:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350634112 00000100:00100000:1.0:1789303084.311188:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350634112 00000100:00100000:1.0:1789303084.311207:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 697 00000100:00100000:1.0:1789303084.311214:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2348:x1876219350633728:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303084.311241:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2348:x1876219350633728:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (1107us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303084.311247:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 697 00000100:00100000:1.0:1789303084.311259:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:1.0:1789303084.311265:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350634112:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303084.311301:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350634112:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 36us (299us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303084.311307:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 698 00000100:00100000:0.0:1789303084.311566:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350633856 00000100:00100000:0.0:1789303084.311571:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350633856 00000100:00100000:0.0:1789303084.311601:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350634368 00000100:00100000:0.0:1789303084.311604:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350634368 00000100:00100000:0.0:1789303084.311622:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 758 00000100:00100000:0.0:1789303084.311628:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350633856:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303084.311662:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbaa00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350633856:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 33us (748us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303084.311668:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 758 00000100:00100000:0.0:1789303084.311682:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 759 00000100:00100000:0.0:1789303084.311689:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350634368:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303084.311726:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbb800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350634368:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 37us (699us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303084.311732:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 759 00000100:00080000:3.0:1789303084.312584:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:3.0:1789303084.312591:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789303084.312594:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00010000:00010000:2.0:1789303084.684740:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303085.709769:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303086.498590:0:27735:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303086.498604:0:27735:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303086.500688:0:27735:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303086.500710:0:27735:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303086.732751:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303087.756703:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789303088.140709:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371231872:0@lo:400:kworker.0 00000100:00100000:2.0:1789303088.140778:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303088.140834:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789303088.140841:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371232000:0@lo:400:kworker.0 00000100:00100000:2.0:1789303088.140865:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303088.141036:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371231744:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141078:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303088.141096:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371232256:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141113:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303088.141126:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789303088.141131:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371231616:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141152:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303088.141168:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371232128:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141183:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303088.141205:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371231872 00000100:00100000:1.0:1789303088.141212:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371231872 00000100:00100000:1.0:1789303088.141253:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371231744 00000100:00100000:1.0:1789303088.141256:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371231744 00000100:00100000:1.0:1789303088.141268:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232128 00000100:00100000:1.0:1789303088.141270:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232128 00000100:00100000:1.0:1789303088.141283:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 699 00000100:00100000:1.0:1789303088.141289:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04cea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371231872:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141337:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04cea300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371231872:12345-0@lo:400:kworker.0 Request processed in 47us (558us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303088.141343:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 699 00000100:00100000:1.0:1789303088.141367:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 700 00000100:00100000:1.0:1789303088.141373:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371231744:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141398:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb9c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371231744:12345-0@lo:400:kworker.0 Request processed in 26us (320us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303088.141404:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 700 00000100:00100000:1.0:1789303088.141413:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 701 00000100:00100000:1.0:1789303088.141418:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b86a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371232128:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141439:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b86a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371232128:12345-0@lo:400:kworker.0 Request processed in 22us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303088.141445:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 701 00000100:00100000:1.0:1789303088.141459:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232000 00000100:00100000:1.0:1789303088.141461:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232000 00000100:00100000:1.0:1789303088.141482:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232256 00000100:00100000:1.0:1789303088.141484:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232256 00000100:00100000:1.0:1789303088.141518:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 760 00000100:00100000:1.0:1789303088.141524:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371232000:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141550:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371232000:12345-0@lo:400:kworker.0 Request processed in 27us (684us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303088.141555:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 760 00000100:00100000:1.0:1789303088.141564:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 761 00000100:00100000:1.0:1789303088.141569:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371232256:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141588:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fb8e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371232256:12345-0@lo:400:kworker.0 Request processed in 20us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303088.141593:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 761 00000100:00080000:1.0:1789303088.141611:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789303088.141614:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789303088.141616:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303088.141649:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371232128:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141658:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371231744:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141665:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371232256:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141690:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371232000:0@lo:400:kworker.0 00000100:00100000:1.0:1789303088.141698:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371231872:0@lo:400:kworker.0 00000100:00100000:0.0:1789303088.143181:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371231616 00000100:00100000:0.0:1789303088.143195:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371231616 00000100:00100000:0.0:1789303088.143252:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 699 00000100:00100000:0.0:1789303088.143262:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371231616:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303088.143347:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371231616:12345-0@lo:400:kworker.0 Request processed in 87us (2193us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303088.143353:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 699 00000100:00100000:0.0:1789303088.143436:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371231616:0@lo:400:kworker.0 00000100:00100000:3.0:1789303088.654319:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1789303088.654354:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25b87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371232384:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789303088.654449:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303088.654617:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232384 00000100:00100000:1.0:1789303088.654629:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232384 00000100:00100000:1.0:1789303088.654688:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:1.0:1789303088.654699:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371232384:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303088.654746:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303088.654803:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371232384:12345-0@lo:13:osp-pre-1-0.0 Request processed in 105us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303088.654810:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1789303088.654905:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25b87480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371232384:0@lo:13:osp-pre-1-0.0 00000040:00080000:3.0:1789303088.765172:0:27786:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303088.765187:0:27786:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303088.767182:0:27786:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303088.767204:0:27786:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303088.781355:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303089.168072:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371232512:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303089.168170:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303089.168265:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232512 00000100:00100000:1.0:1789303089.168277:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232512 00000100:00100000:1.0:1789303089.168392:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 297 00000100:00100000:1.0:1789303089.168405:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371232512:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789303089.168451:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303089.173113:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e0267f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371232512:12345-0@lo:13:osp-pre-0-0.0 Request processed in 4693us (4926us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303089.173205:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 297 00000100:00100000:3.0:1789303089.173266:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce9180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371232512:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303089.422659:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303089.422721:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303089.422752:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303089.422803:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303089.422831:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303089.422854:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303089.422883:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:1.0:1789303089.422919:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350634496 00000100:00100000:1.0:1789303089.422932:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350634496 00000100:00100000:1.0:1789303089.422995:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 700 00000100:00100000:1.0:1789303089.423007:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350634496:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303089.423107:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a2450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350634496:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 102us (302us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303089.423115:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 700 00000100:00100000:1.0:1789303089.423154:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350634880 00000100:00100000:1.0:1789303089.423158:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350634880 00000100:00100000:1.0:1789303089.423189:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635136 00000100:00100000:1.0:1789303089.423192:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635136 00000100:00100000:1.0:1789303089.423210:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350634752 00000100:00100000:1.0:1789303089.423213:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350634752 00000100:00100000:1.0:1789303089.423225:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635264 00000100:00100000:1.0:1789303089.423227:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635264 00000100:00100000:0.0:1789303089.423344:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 762 00000100:00100000:0.0:1789303089.423355:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350634880:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303089.423404:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350634880:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 52us (683us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303089.423411:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 762 00000100:00100000:0.0:1789303089.423426:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 763 00000100:00100000:0.0:1789303089.423432:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350635136:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303089.423460:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87100 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350635136:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 28us (708us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303089.423467:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 763 00000100:00100000:0.0:1789303089.423479:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 764 00000100:00100000:0.0:1789303089.423485:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350634752:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303089.423554:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b86d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350634752:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (722us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303089.423561:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 764 00000100:00100000:0.0:1789303089.423762:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350634624 00000100:00100000:0.0:1789303089.423768:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350634624 00000100:00100000:0.0:1789303089.423802:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635008 00000100:00100000:0.0:1789303089.423805:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635008 00000100:00100000:0.0:1789303089.423830:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 702 00000100:00100000:0.0:1789303089.423837:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2346:x1876219350634624:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303089.423880:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b84e00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2346:x1876219350634624:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 44us (1236us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303089.423888:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 702 00000100:00100000:0.0:1789303089.423902:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 703 00000100:00100000:0.0:1789303089.423908:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25b87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2346:x1876219350635008:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303089.423930:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25b87480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2346:x1876219350635008:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (1076us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303089.423936:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 703 00000100:00080000:0.0:1789303089.423971:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789303089.423978:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303089.423980:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:1.0:1789303089.426656:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 765 00000100:00100000:1.0:1789303089.426681:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350635264:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303089.426806:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef53b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350635264:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 135us (3921us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303089.426814:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 765 00010000:00010000:3.0:1789303089.807118:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303090.381388:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303090.381402:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303090.384229:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303090.384251:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303090.829078:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303090.975681:0:27837:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303090.975696:0:27837:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303090.978870:0:27837:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303090.978895:0:27837:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303091.856974:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1789303092.877662:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303093.109761:0:27888:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303093.109969:0:27888:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303093.111624:0:27888:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303093.111646:0:27888:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:3.0:1789303093.270440:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371232768:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.270563:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303093.270626:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371233280:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.270657:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303093.270688:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1789303093.270694:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371232640:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271060:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303093.271104:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371233152:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271129:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303093.271160:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371232896:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271186:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303093.271206:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303093.271211:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371233024:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271234:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303093.271270:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232768 00000100:00100000:3.0:1789303093.271280:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232768 00000100:00100000:3.0:1789303093.271332:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233152 00000100:00100000:3.0:1789303093.271335:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233152 00000100:00100000:3.0:1789303093.271366:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232896 00000100:00100000:3.0:1789303093.271369:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232896 00000100:00100000:3.0:1789303093.271387:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 704 00000100:00100000:3.0:1789303093.271396:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371232768:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271459:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371232768:12345-0@lo:400:kworker.0 Request processed in 64us (897us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303093.271467:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 704 00000100:00100000:3.0:1789303093.271485:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 705 00000100:00100000:3.0:1789303093.271491:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371233152:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271545:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63c000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371233152:12345-0@lo:400:kworker.0 Request processed in 54us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303093.271553:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 705 00000100:00100000:3.0:1789303093.271565:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 706 00000100:00100000:3.0:1789303093.271570:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371232896:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271599:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63df80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371232896:12345-0@lo:400:kworker.0 Request processed in 29us (413us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303093.271604:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 706 00000100:00100000:3.0:1789303093.271631:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233280 00000100:00100000:3.0:1789303093.271634:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233280 00000100:00100000:3.0:1789303093.271662:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233024 00000100:00100000:3.0:1789303093.271666:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233024 00000100:00100000:3.0:1789303093.271686:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 766 00000100:00100000:3.0:1789303093.271693:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371233280:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271738:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371233280:12345-0@lo:400:kworker.0 Request processed in 46us (1080us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303093.271745:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 766 00000100:00100000:3.0:1789303093.271759:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 767 00000100:00100000:3.0:1789303093.271765:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371233024:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271790:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371233024:12345-0@lo:400:kworker.0 Request processed in 26us (556us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303093.271796:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 767 00000100:00080000:3.0:1789303093.271854:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:3.0:1789303093.271859:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:3.0:1789303093.271862:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:3.0:1789303093.271905:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef43800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371233152:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271917:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371232768:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271931:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63ca80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371233280:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271967:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371233024:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.271976:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef42300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371232896:0@lo:400:kworker.0 00000100:00100000:0.0:1789303093.274364:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371232640 00000100:00100000:0.0:1789303093.274382:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371232640 00000100:00100000:0.0:1789303093.274448:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 701 00000100:00100000:0.0:1789303093.274459:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371232640:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303093.274589:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371232640:12345-0@lo:400:kworker.0 Request processed in 131us (3528us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303093.274597:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 701 00000100:00100000:3.0:1789303093.274890:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e0267d500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371232640:0@lo:400:kworker.0 00000100:00100000:3.0:1789303093.774386:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371233408:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789303093.774519:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303093.774605:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233408 00000100:00100000:0.0:1789303093.774615:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233408 00000100:00100000:0.0:1789303093.774700:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1789303093.774724:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371233408:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1789303093.774779:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:0.0:1789303093.775208:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371233408:12345-0@lo:13:osp-pre-1-0.0 Request processed in 484us (708us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303093.775217:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:3.0:1789303093.775292:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef50700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371233408:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303093.900825:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1789303094.285475:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1789303094.285559:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371233536:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303094.285641:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303094.285754:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233536 00000100:00100000:2.0:1789303094.285765:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233536 00000100:00100000:2.0:1789303094.285824:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 299 00000100:00100000:2.0:1789303094.285835:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371233536:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303094.285896:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303094.285949:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371233536:12345-0@lo:13:osp-pre-0-0.0 Request processed in 116us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303094.285956:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 299 00000100:00100000:2.0:1789303094.286003:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960e04ce8380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371233536:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303094.542493:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303094.542559:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303094.542587:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303094.542607:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303094.542625:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303094.542658:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303094.542686:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303094.542713:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635904 00000100:00100000:0.0:1789303094.542722:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635904 00000100:00100000:0.0:1789303094.542777:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635520 00000100:00100000:0.0:1789303094.542779:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635520 00000100:00100000:0.0:1789303094.542801:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 707 00000100:00100000:0.0:1789303094.542810:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2346:x1876219350635904:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303094.542879:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adc00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2346:x1876219350635904:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 69us (290us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303094.542896:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 707 00000100:00100000:0.0:1789303094.542914:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:0.0:1789303094.542921:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2346:x1876219350635520:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303094.542945:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac000 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2346:x1876219350635520:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (319us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303094.542951:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:0.0:1789303094.542969:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635648 00000100:00100000:0.0:1789303094.542973:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635648 00000100:00100000:0.0:1789303094.542997:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636160 00000100:00100000:0.0:1789303094.543000:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636160 00000100:00100000:0.0:1789303094.543024:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635776 00000100:00100000:0.0:1789303094.543027:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635776 00000100:00100000:0.0:1789303094.543040:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636032 00000100:00100000:0.0:1789303094.543042:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636032 00000100:00100000:0.0:1789303094.543056:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 768 00000100:00100000:0.0:1789303094.543061:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350635648:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303094.543086:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350635648:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (527us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303094.543093:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 768 00000100:00100000:0.0:1789303094.543104:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 769 00000100:00100000:0.0:1789303094.543111:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350636160:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303094.543132:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2346:x1876219350636160:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (524us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303094.543141:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 769 00000100:00100000:0.0:1789303094.543151:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 770 00000100:00100000:0.0:1789303094.543158:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350635776:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303094.543180:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2346:x1876219350635776:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 23us (522us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303094.543185:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 770 00000100:00100000:0.0:1789303094.543195:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 771 00000100:00100000:0.0:1789303094.543201:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350636032:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303094.543222:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2346:x1876219350636032:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (535us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303094.543227:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 771 00000100:00080000:0.0:1789303094.543252:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789303094.543255:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303094.543257:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789303094.547783:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350635392 00000100:00100000:2.0:1789303094.547797:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350635392 00000100:00100000:2.0:1789303094.547846:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 702 00000100:00100000:2.0:1789303094.547855:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350635392:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303094.547923:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a3c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2346:x1876219350635392:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 68us (5437us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303094.547930:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 702 00010000:00010000:2.0:1789303094.924743:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303095.008230:0:27940:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303095.008239:0:27940:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303095.008840:0:27940:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303095.008850:0:27940:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303095.949081:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303096.973737:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303097.253625:0:27991:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303097.253634:0:27991:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303097.267325:0:27991:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303097.267347:0:27991:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303097.996877:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789303098.380681:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371233664:0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.380709:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371233920:0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.380759:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303098.380760:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303098.380830:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3f63d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371233792:0@lo:400:kworker.0 00000100:00100000:1.0:1789303098.380853:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233664 00000100:00100000:1.0:1789303098.380861:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233664 00000100:00100000:1.0:1789303098.380902:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 703 00000100:00100000:1.0:1789303098.380909:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371233664:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303098.388721:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371234176:0@lo:400:kworker.0 00000100:00100000:0.0:1789303098.388828:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303098.388878:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233920 00000100:00100000:0.0:1789303098.388886:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233920 00000100:00100000:0.0:1789303098.388932:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234176 00000100:00100000:0.0:1789303098.388935:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234176 00000100:00100000:0.0:1789303098.388957:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 709 00000100:00100000:0.0:1789303098.388965:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371233920:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303098.389035:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63ce00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371233920:12345-0@lo:400:kworker.0 Request processed in 70us (8279us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303098.389041:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 709 00000100:00100000:0.0:1789303098.389058:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 710 00000100:00100000:0.0:1789303098.389064:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371234176:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303098.389094:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6adf80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371234176:12345-0@lo:400:kworker.0 Request processed in 32us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303098.389100:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 710 00000100:00080000:0.0:1789303098.389120:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303098.389151:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371234176:0@lo:400:kworker.0 00000100:00100000:0.0:1789303098.389185:0:3302:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1789303098.389192:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371234304:0@lo:400:kworker.0 00000100:00100000:0.0:1789303098.389219:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303098.389246:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234304 00000100:00100000:0.0:1789303098.389249:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234304 00000100:00100000:0.0:1789303098.389279:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 772 00000100:00100000:0.0:1789303098.389285:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371234304:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303098.389319:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3302:x1876219371234304:12345-0@lo:400:kworker.0 Request processed in 34us (99us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303098.389324:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 772 00000100:00080000:0.0:1789303098.389339:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789303098.389357:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6aea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3302:1876219371234304:0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.389428:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371234048:0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.389463:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303098.389653:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234048 00000100:00100000:3.0:1789303098.389657:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234048 00000100:00100000:3.0:1789303098.389687:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 773 00000100:00100000:3.0:1789303098.389694:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371234048:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.389729:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63e680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371234048:12345-0@lo:400:kworker.0 Request processed in 36us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303098.389734:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 773 00000100:00100000:3.0:1789303098.389763:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371234048:0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.389784:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63f800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371233920:0@lo:400:kworker.0 00000100:00100000:2.0:1789303098.390607:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1789303098.391118:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:3.0:1789303098.392383:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371233792 00000100:00100000:3.0:1789303098.392395:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371233792 00000100:00100000:3.0:1789303098.392450:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 711 00000100:00100000:3.0:1789303098.392459:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371233792:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.392622:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce8380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371233792:12345-0@lo:400:kworker.0 Request processed in 163us (2016us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303098.392630:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 711 00000100:00100000:0.0:1789303098.393213:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f334a1450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371233664:12345-0@lo:400:kworker.0 Request processed in 12301us (12453us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303098.393223:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 703 00000100:00100000:1.0:1789303098.393406:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63ea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371233664:0@lo:400:kworker.0 00000100:00100000:2.0:1789303098.393444:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3f63d880 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371233792:0@lo:400:kworker.0 00000100:00100000:3.0:1789303098.924751:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371234432:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789303098.924818:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303098.924898:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234432 00000100:00100000:3.0:1789303098.925135:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234432 00000100:00100000:3.0:1789303098.925180:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:3.0:1789303098.925187:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371234432:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789303098.925221:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789303098.925262:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371234432:12345-0@lo:13:osp-pre-1-0.0 Request processed in 76us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303098.925269:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00100000:3.0:1789303098.925306:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371234432:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789303099.021713:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303099.267185:0:28042:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303099.267195:0:28042:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303099.267913:0:28042:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303099.267922:0:28042:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:1.0:1789303099.407284:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789303099.407307:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371234560:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303099.407372:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303099.407444:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234560 00000100:00100000:1.0:1789303099.407451:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234560 00000100:00100000:1.0:1789303099.407494:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 301 00000100:00100000:1.0:1789303099.407518:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371234560:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789303099.407550:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303099.407593:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371234560:12345-0@lo:13:osp-pre-0-0.0 Request processed in 90us (220us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303099.407600:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 301 00000100:00100000:1.0:1789303099.407642:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371234560:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1789303099.661555:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303099.661609:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636288 00000100:00100000:0.0:1789303099.661620:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636288 00000100:00100000:0.0:1789303099.661684:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 704 00000100:00100000:0.0:1789303099.661694:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350636288:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303099.661780:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538450 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350636288:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 78us (225us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303099.661788:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 704 00000100:00100000:2.0:1789303099.672538:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303099.672574:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303099.672617:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303099.672816:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636672 00000100:00100000:3.0:1789303099.672827:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636672 00000100:00100000:3.0:1789303099.672884:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636928 00000100:00100000:3.0:1789303099.672887:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636928 00000100:00100000:3.0:1789303099.672908:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 774 00000100:00100000:3.0:1789303099.672917:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350636672:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303099.672998:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350636672:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 81us (423us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303099.673006:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 774 00000100:00100000:3.0:1789303099.673023:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 775 00000100:00100000:3.0:1789303099.673030:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350636928:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303099.673071:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbed80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350636928:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 41us (453us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303099.673076:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 775 00000100:00080000:2.0:1789303099.674567:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303099.674575:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789303099.675534:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636416 00000100:00100000:0.0:1789303099.675545:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636416 00000100:00100000:0.0:1789303099.675621:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 712 00000100:00100000:0.0:1789303099.675641:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960e04ce9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2347:x1876219350636416:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303099.675941:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960e04ce9180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2347:x1876219350636416:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 311us (3411us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303099.675949:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 712 00000100:00080000:3.0:1789303099.676690:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303099.679553:0:3293:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:3.0:1789303099.679643:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636544 00000100:00100000:3.0:1789303099.679653:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636544 00000100:00100000:3.0:1789303099.679747:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 776 00000100:00100000:3.0:1789303099.679758:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f08598c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350636544:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303099.680176:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f08598c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350636544:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 418us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303099.680184:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 776 00000100:00080000:2.0:1789303099.680217:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789303099.681356:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303099.681806:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303099.682351:0:13439:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637056 00000100:00100000:2.0:1789303099.682361:0:13439:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637056 00000100:00100000:3.0:1789303099.682443:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350636800 00000100:00100000:3.0:1789303099.682450:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350636800 00000100:00100000:3.0:1789303099.682492:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 713 00000100:00100000:3.0:1789303099.682524:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350636800:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303099.682684:0:13439:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 777 00000100:00100000:2.0:1789303099.682694:0:13439:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350637056:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303099.682739:0:13439:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350637056:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 46us (933us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303099.682747:0:13439:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 777 00000100:00080000:0.0:1789303099.682957:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:0.0:1789303099.682963:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:3.0:1789303099.683349:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6aea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350636800:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 847us (1995us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303099.683359:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 713 00010000:00010000:3.0:1789303100.047202:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303100.621281:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303100.621291:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303100.621995:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303100.622005:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303101.081718:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303101.293159:0:28094:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303101.293171:0:28094:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303101.293789:0:28094:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303101.293801:0:28094:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:2.0:1789303102.092797:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303103.116766:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303103.350499:0:28145:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303103.350528:0:28145:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303103.353682:0:28145:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303103.353699:0:28145:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000100:00100000:0.0:1789303103.502281:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371234816:0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.502383:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303103.502436:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371235328:0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.502462:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303103.502490:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1789303103.502496:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371234688:0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.502543:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303103.502564:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371235200:0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.502749:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371234944:0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.502780:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303103.502800:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1789303103.502806:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371235072:0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.502826:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303103.502867:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234816 00000100:00100000:1.0:1789303103.502875:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234816 00000100:00100000:1.0:1789303103.502910:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234944 00000100:00100000:1.0:1789303103.502914:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234944 00000100:00100000:1.0:1789303103.502943:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 714 00000100:00100000:1.0:1789303103.502964:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371234816:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.503010:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371234816:12345-0@lo:400:kworker.0 Request processed in 47us (627us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303103.503016:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 714 00000100:00100000:1.0:1789303103.503032:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 715 00000100:00100000:1.0:1789303103.503038:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371234944:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.503065:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef52300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3300:x1876219371234944:12345-0@lo:400:kworker.0 Request processed in 28us (284us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303103.503069:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 715 00000100:00100000:1.0:1789303103.503086:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235328 00000100:00100000:1.0:1789303103.503090:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235328 00000100:00100000:1.0:1789303103.503112:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235072 00000100:00100000:1.0:1789303103.503114:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235072 00000100:00100000:1.0:1789303103.503130:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 778 00000100:00100000:1.0:1789303103.503135:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371235328:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.503164:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2a00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371235328:12345-0@lo:400:kworker.0 Request processed in 28us (701us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303103.503169:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 778 00000100:00100000:1.0:1789303103.503180:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 779 00000100:00100000:1.0:1789303103.503185:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371235072:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.503210:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef50700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371235072:12345-0@lo:400:kworker.0 Request processed in 25us (384us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303103.503225:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 779 00000100:00080000:1.0:1789303103.503243:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789303103.503247:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:1.0:1789303103.503250:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303103.503587:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371235072:0@lo:400:kworker.0 00000100:00100000:1.0:1789303103.503599:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af800 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3300:1876219371234944:0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.503916:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303103.503967:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac700 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371234816:0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.503977:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6af480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371235328:0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.504001:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235200 00000100:00100000:0.0:1789303103.504004:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235200 00000100:00100000:0.0:1789303103.504034:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 716 00000100:00100000:0.0:1789303103.504040:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371235200:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.504075:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371235200:12345-0@lo:400:kworker.0 Request processed in 35us (159us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303103.504081:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 716 00000100:00100000:0.0:1789303103.504102:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371234688 00000100:00100000:0.0:1789303103.504106:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371234688 00000100:00100000:0.0:1789303103.504127:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 705 00000100:00100000:0.0:1789303103.504133:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371234688:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.504161:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371234688:12345-0@lo:400:kworker.0 Request processed in 28us (1618us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303103.504168:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 705 00000100:00100000:0.0:1789303103.504195:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371234688:0@lo:400:kworker.0 00000100:00100000:0.0:1789303103.504205:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371235200:0@lo:400:kworker.0 00000100:00080000:1.0:1789303103.505692:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789303104.012982:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371235456:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1789303104.013056:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1789303104.013321:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235456 00000100:00100000:3.0:1789303104.013329:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235456 00000100:00100000:3.0:1789303104.013383:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:3.0:1789303104.013392:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371235456:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1789303104.013431:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:3.0:1789303104.019123:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc380 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371235456:12345-0@lo:13:osp-pre-1-0.0 Request processed in 5718us (6053us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303104.019139:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:2.0:1789303104.019376:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371235456:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789303104.140825:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789303104.526592:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1789303104.526630:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371235584:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789303104.526701:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303104.527033:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235584 00000100:00100000:1.0:1789303104.527052:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235584 00000100:00100000:1.0:1789303104.527117:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 303 00000100:00100000:1.0:1789303104.527125:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371235584:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1789303104.527169:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:1.0:1789303104.527218:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbc700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371235584:12345-0@lo:13:osp-pre-0-0.0 Request processed in 93us (517us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303104.527228:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 303 00000100:00100000:3.0:1789303104.528580:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371235584:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303104.794218:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303104.794268:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303104.794299:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303104.794324:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303104.794346:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303104.794363:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303104.794402:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303104.794428:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637184 00000100:00100000:0.0:1789303104.794438:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637184 00000100:00100000:0.0:1789303104.794488:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 706 00000100:00100000:0.0:1789303104.794498:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350637184:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303104.794589:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353b850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2348:x1876219350637184:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 91us (263us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303104.794596:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 706 00000100:00100000:1.0:1789303104.794752:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637312 00000100:00100000:1.0:1789303104.794757:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637312 00000100:00100000:1.0:1789303104.794815:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 717 00000100:00100000:1.0:1789303104.794823:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2348:x1876219350637312:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303104.794857:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2348:x1876219350637312:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 35us (648us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303104.794863:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 717 00000100:00100000:1.0:1789303104.794874:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637696 00000100:00100000:1.0:1789303104.794877:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637696 00000100:00100000:1.0:1789303104.794899:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 718 00000100:00100000:1.0:1789303104.794905:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350637696:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303104.794950:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1180 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2348:x1876219350637696:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 43us (585us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303104.794957:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 718 00000100:00100000:1.0:1789303104.794976:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637568 00000100:00100000:1.0:1789303104.794980:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637568 00000100:00100000:1.0:1789303104.795003:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637824 00000100:00100000:1.0:1789303104.795006:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637824 00000100:00100000:1.0:1789303104.795024:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637440 00000100:00100000:1.0:1789303104.795027:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637440 00000100:00100000:1.0:1789303104.795039:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350637952 00000100:00100000:1.0:1789303104.795042:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350637952 00000100:00100000:1.0:1789303104.795054:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 780 00000100:00100000:1.0:1789303104.795060:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350637568:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303104.795085:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1880 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350637568:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 25us (827us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303104.795092:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 780 00000100:00100000:1.0:1789303104.795104:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 781 00000100:00100000:1.0:1789303104.795110:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350637824:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303104.795129:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0a80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2348:x1876219350637824:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (831us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303104.795136:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 781 00000100:00100000:1.0:1789303104.795146:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 782 00000100:00100000:1.0:1789303104.795151:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350637440:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303104.795171:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0e00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2348:x1876219350637440:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (824us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303104.795177:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 782 00000100:00100000:1.0:1789303104.795188:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 783 00000100:00100000:1.0:1789303104.795193:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350637952:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303104.795213:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e1f80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2348:x1876219350637952:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 19us (810us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303104.795219:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 783 00000100:00080000:0.0:1789303104.795951:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303104.795957:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:0.0:1789303104.795960:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789303105.168124:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:3.0:1789303105.540167:0:28196:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303105.540175:0:28196:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303105.541171:0:28196:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303105.541184:0:28196:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303106.190614:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303107.215742:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:1.0:1789303108.206058:0:28248:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:1.0:1789303108.206072:0:28248:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:1.0:1789303108.207709:0:28248:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:1.0:1789303108.207726:0:28248:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303108.237794:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1789303108.622076:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371235840:0@lo:400:kworker.0 00000100:00100000:0.0:1789303108.622120:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371235712:0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622144:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303108.622172:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303108.622218:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371236352:0@lo:400:kworker.0 00000100:00100000:0.0:1789303108.622224:0:3302:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371236224:0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622249:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303108.622260:0:3302:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303108.622282:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371236096:0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622306:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303108.622320:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1789303108.622325:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371235968:0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622349:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235840 00000100:00100000:1.0:1789303108.622355:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235840 00000100:00100000:1.0:1789303108.622396:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 719 00000100:00100000:1.0:1789303108.622405:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371235840:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622449:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af480 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371235840:12345-0@lo:400:kworker.0 Request processed in 45us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303108.622456:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 719 00000100:00100000:1.0:1789303108.622478:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236352 00000100:00100000:1.0:1789303108.622481:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236352 00000100:00100000:1.0:1789303108.622536:0:6695:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236096 00000100:00100000:1.0:1789303108.622540:0:6695:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236096 00000100:00100000:1.0:1789303108.622578:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 784 00000100:00100000:1.0:1789303108.622587:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371236352:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303108.622588:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235712 00000100:00100000:0.0:1789303108.622595:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235712 00000100:00100000:1.0:1789303108.622623:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371236352:12345-0@lo:400:kworker.0 Request processed in 39us (374us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303108.622631:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 707 00000100:00100000:1.0:1789303108.622635:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 784 00000100:00100000:0.0:1789303108.622644:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371235712:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622651:0:6695:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 785 00000100:00100000:1.0:1789303108.622660:0:6695:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371236096:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303108.622694:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538c50 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3302:x1876219371235712:12345-0@lo:400:kworker.0 Request processed in 53us (522us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303108.622704:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 707 00000100:00100000:1.0:1789303108.622710:0:6695:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6af800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371236096:12345-0@lo:400:kworker.0 Request processed in 40us (391us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303108.622719:0:6695:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 785 00000100:00100000:0.0:1789303108.622735:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236224 00000100:00100000:0.0:1789303108.622740:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236224 00000100:00080000:1.0:1789303108.622747:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:1.0:1789303108.622754:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00080000:1.0:1789303108.622757:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:0.0:1789303108.622772:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 720 00000100:00100000:0.0:1789303108.622781:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371236224:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622794:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef53b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371235840:0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622808:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ac380 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371236352:0@lo:400:kworker.0 00000100:00100000:0.0:1789303108.622820:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3302:x1876219371236224:12345-0@lo:400:kworker.0 Request processed in 40us (560us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303108.622828:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 720 00000100:00100000:1.0:1789303108.622854:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1789303108.622865:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:4acabb58-ab41-4250-a3d6-ebb667f59134:3302:1876219371235712:0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622883:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef52680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371236096:0@lo:400:kworker.0 00000100:00100000:0.0:1789303108.622886:0:3302:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ad180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3302:1876219371236224:0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622905:0:5843:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371235968 00000100:00100000:1.0:1789303108.622909:0:5843:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371235968 00000100:00100000:1.0:1789303108.622944:0:5843:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 721 00000100:00100000:1.0:1789303108.622950:0:5843:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371235968:12345-0@lo:400:kworker.0 00000100:00100000:1.0:1789303108.622983:0:5843:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ac380 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371235968:12345-0@lo:400:kworker.0 Request processed in 33us (130us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303108.622989:0:5843:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 721 00000100:00080000:1.0:1789303108.623001:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303108.623014:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51f80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371235968:0@lo:400:kworker.0 00000100:00100000:3.0:1789303109.133285:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371236480:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1789303109.133355:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303109.133563:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236480 00000100:00100000:2.0:1789303109.133571:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236480 00000100:00100000:2.0:1789303109.133631:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1789303109.133640:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371236480:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1789303109.133672:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:2.0:1789303109.133722:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63d500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371236480:12345-0@lo:13:osp-pre-1-0.0 Request processed in 83us (367us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303109.133730:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:0.0:1789303109.134235:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f1e6ace00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371236480:0@lo:13:osp-pre-1-0.0 00010000:00010000:3.0:1789303109.261229:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1789303109.645403:0:3300:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1789303109.645425:0:3300:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371236608:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303109.645490:0:3300:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303109.645654:0:6698:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236608 00000100:00100000:2.0:1789303109.645661:0:6698:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236608 00000100:00100000:2.0:1789303109.645711:0:6698:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 305 00000100:00100000:2.0:1789303109.645720:0:6698:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371236608:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303109.645751:0:6698:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303109.645794:0:6698:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e0000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3300:x1876219371236608:12345-0@lo:13:osp-pre-0-0.0 Request processed in 75us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303109.645801:0:6698:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 305 00000100:00100000:2.0:1789303109.645840:0:3300:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbdc00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3300:1876219371236608:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1789303109.900067:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303109.900102:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303109.900127:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:0.0:1789303109.900163:0:5845:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638208 00000100:00100000:0.0:1789303109.900172:0:5845:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638208 00000100:00100000:0.0:1789303109.900229:0:5845:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 722 00000100:00100000:0.0:1789303109.900240:0:5845:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2347:x1876219350638208:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303109.900318:0:5845:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e2d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2347:x1876219350638208:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 81us (270us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303109.900326:0:5845:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 722 00000100:00080000:0.0:1789303109.900356:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303109.900366:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638464 00000100:00100000:0.0:1789303109.900369:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638464 00000100:00100000:0.0:1789303109.900394:0:8571:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638720 00000100:00100000:0.0:1789303109.900396:0:8571:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638720 00000100:00100000:0.0:1789303109.900414:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 786 00000100:00100000:0.0:1789303109.900420:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350638464:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303109.900446:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350638464:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 26us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303109.900453:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 786 00000100:00100000:0.0:1789303109.900463:0:8571:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 787 00000100:00100000:0.0:1789303109.900469:0:8571:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350638720:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303109.900491:0:8571:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3a9e3b80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350638720:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (363us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303109.900497:0:8571:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 787 00000100:00080000:0.0:1789303109.900552:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:0.0:1789303109.900555:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00100000:2.0:1789303109.907471:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303109.907532:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303109.907557:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303109.907582:0:3294:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303109.907606:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638336 00000100:00100000:2.0:1789303109.907615:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638336 00000100:00100000:2.0:1789303109.907658:0:6696:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638848 00000100:00100000:2.0:1789303109.907661:0:6696:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638848 00000100:00100000:2.0:1789303109.907680:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 788 00000100:00100000:2.0:1789303109.907689:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350638336:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303109.907748:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbdc00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350638336:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 61us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303109.907754:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 788 00000100:00100000:2.0:1789303109.907770:0:6696:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 789 00000100:00100000:2.0:1789303109.907775:0:6696:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350638848:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303109.907796:0:6696:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbca80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350638848:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303109.907802:0:6696:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 789 00000100:00080000:2.0:1789303109.907819:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303109.907823:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789303109.908079:0:5837:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638080 00000100:00100000:0.0:1789303109.908084:0:5837:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638080 00000100:00100000:0.0:1789303109.908111:0:5837:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:0.0:1789303109.908117:0:5837:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350638080:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303109.908146:0:5837:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33539050 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0001:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350638080:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 29us (683us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303109.908153:0:5837:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 708 00000100:00100000:3.0:1789303109.908835:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638592 00000100:00100000:3.0:1789303109.908840:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638592 00000100:00100000:3.0:1789303109.908884:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 723 00000100:00100000:3.0:1789303109.908900:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2347:x1876219350638592:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:3.0:1789303109.908955:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd880 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2347:x1876219350638592:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 64us (1397us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303109.908962:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 723 00000100:00080000:2.0:1789303109.909348:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00010000:00010000:3.0:1789303110.306386:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1789303110.863583:0:11693:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:2.0:1789303110.863592:0:11693:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:2.0:1789303110.864750:0:11693:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:2.0:1789303110.864762:0:11693:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00000040:00080000:3.0:1789303110.866851:0:28300:0:(llog.c:805:llog_process_thread()) index: 1, lh_last_idx: 1 synced_idx: 1 lgh_last_idx: 1 00000040:00080000:3.0:1789303110.866858:0:28300:0:(llog_cat.c:163:llog_cat_process_common()) processing log [0x1:0x8:0x0] at index 1 of catalog [0xa:0x7:0x0] 00000040:00080000:3.0:1789303110.869330:0:28300:0:(llog.c:862:llog_process_thread()) stop processing plain [0x1:0x8:0x0] index 64768 count 1 00000040:00080000:3.0:1789303110.869349:0:28300:0:(llog.c:862:llog_process_thread()) stop processing catalog [0xa:0x7:0x0] index 2 count 2 00010000:00010000:3.0:1789303111.308739:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303112.332804:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1789303113.380251:0:5832:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1789303113.740800:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1789303113.740824:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371236992:0@lo:400:kworker.0 00000100:00100000:2.0:1789303113.740905:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303113.740989:0:18427:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236992 00000100:00100000:2.0:1789303113.740996:0:18427:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236992 00000100:00100000:2.0:1789303113.741049:0:18427:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 724 00000100:00100000:2.0:1789303113.741058:0:18427:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371236992:12345-0@lo:400:kworker.0 00000100:00100000:2.0:1789303113.741106:0:18427:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbe300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3301:x1876219371236992:12345-0@lo:400:kworker.0 Request processed in 49us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303113.741112:0:18427:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 724 00000100:00080000:2.0:1789303113.741133:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:2.0:1789303113.741162:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3301:1876219371236992:0@lo:400:kworker.0 00000100:00100000:2.0:1789303113.741197:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371237120:0@lo:400:kworker.0 00000100:00100000:2.0:1789303113.741222:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303113.741632:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371236864:0@lo:400:kworker.0 00000100:00100000:1.0:1789303113.741664:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303113.741695:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371237376:0@lo:400:kworker.0 00000100:00100000:1.0:1789303113.741716:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303113.741733:0:3303:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1789303113.741738:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371236736:0@lo:400:kworker.0 00000100:00100000:1.0:1789303113.741761:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303113.741778:0:3303:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371237248:0@lo:400:kworker.0 00000100:00100000:1.0:1789303113.741795:0:3303:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303113.741815:0:5838:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236736 00000100:00100000:1.0:1789303113.741818:0:5838:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236736 00000100:00100000:1.0:1789303113.741842:0:5838:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 709 00000100:00100000:1.0:1789303113.741848:0:5838:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371236736:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303113.741943:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371236864 00000100:00100000:3.0:1789303113.741960:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371236864 00000100:00100000:3.0:1789303113.741986:0:6919:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371237248 00000100:00100000:3.0:1789303113.741989:0:6919:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371237248 00000100:00100000:3.0:1789303113.742009:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 725 00000100:00100000:3.0:1789303113.742016:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371236864:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303113.742054:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ace00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3303:x1876219371236864:12345-0@lo:400:kworker.0 Request processed in 38us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303113.742060:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 725 00000100:00100000:3.0:1789303113.742071:0:6919:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 726 00000100:00100000:3.0:1789303113.742077:0:6919:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371237248:12345-0@lo:400:kworker.0 00000100:00100000:3.0:1789303113.742099:0:6919:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ef51f80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3303:x1876219371237248:12345-0@lo:400:kworker.0 Request processed in 23us (303us total) trans 0 rc 0/0 00000100:00100000:3.0:1789303113.742104:0:6919:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 726 00000100:00100000:3.0:1789303113.742132:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbfb80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3303:1876219371237248:0@lo:400:kworker.0 00000100:00100000:3.0:1789303113.742140:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3303:1876219371236864:0@lo:400:kworker.0 00000100:00100000:1.0:1789303113.742284:0:5838:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f33538850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:4acabb58-ab41-4250-a3d6-ebb667f59134+19:3303:x1876219371236736:12345-0@lo:400:kworker.0 Request processed in 435us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303113.742290:0:5838:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 709 00000100:00100000:3.0:1789303113.742636:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbea00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:4acabb58-ab41-4250-a3d6-ebb667f59134:3303:1876219371236736:0@lo:400:kworker.0 00000100:00100000:0.0:1789303113.742992:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371237120 00000100:00100000:0.0:1789303113.742998:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371237120 00000100:00100000:0.0:1789303113.743036:0:6697:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371237376 00000100:00100000:0.0:1789303113.743039:0:6697:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371237376 00000100:00100000:0.0:1789303113.743066:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 790 00000100:00100000:0.0:1789303113.743073:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371237120:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303113.743121:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371237120:12345-0@lo:400:kworker.0 Request processed in 48us (1897us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303113.743127:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 790 00000100:00100000:0.0:1789303113.743140:0:6697:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 791 00000100:00100000:0.0:1789303113.743146:0:6697:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371237376:12345-0@lo:400:kworker.0 00000100:00100000:0.0:1789303113.743177:0:6697:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f1e6ad180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0000-mdtlov_UUID+5:3303:x1876219371237376:12345-0@lo:400:kworker.0 Request processed in 31us (1460us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303113.743182:0:6697:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 791 00000100:00100000:0.0:1789303113.743210:0:3303:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf480 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3303:1876219371237376:0@lo:400:kworker.0 00000100:00100000:0.0:1789303113.743232:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbf100 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371237120:0@lo:400:kworker.0 00000100:00080000:2.0:1789303113.743670:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303113.743675:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00080000:2.0:1789303113.743677:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:0.0:1789303114.258078:0:3301:0:(ptlrpcd.c:344:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1789303114.258103:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371237504:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1789303114.258169:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1789303114.258390:0:6699:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371237504 00000100:00100000:1.0:1789303114.258400:0:6699:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371237504 00000100:00100000:1.0:1789303114.258451:0:6699:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:1.0:1789303114.258460:0:6699:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371237504:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1789303114.258494:0:6699:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3856662528 avail 3854565376 00000100:00100000:1.0:1789303114.258571:0:6699:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3ffa7800 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371237504:12345-0@lo:13:osp-pre-1-0.0 Request processed in 111us (401us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303114.258579:0:6699:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1789303114.258719:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f3ef51180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371237504:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1789303114.381938:0:17746:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:02000400:2.0:1789303114.460172:0:28350:0:(debug.c:653:libcfs_debug_mark_buffer()) DEBUG MARKER: sanity-hsm test_12u: @@@@@@ FAIL: request on 0x200000402:0x37:0x0 is not SUCCEED on mds1 00000100:00100000:3.0:1789303114.770117:0:3301:0:(client.c:1920:ptlrpc_send_new_req()) Sending RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371237632:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1789303114.770187:0:3301:0:(events.c:344:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1789303114.770808:0:26464:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219371237632 00000100:00100000:2.0:1789303114.770816:0:26464:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219371237632 00000100:00100000:2.0:1789303114.770892:0:26464:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 307 00000100:00100000:2.0:1789303114.770900:0:26464:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371237632:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1789303114.770936:0:26464:0:(ofd_obd.c:735:ofd_statfs()) blocks cached 0 granted 98631680 pending 0 free 3858759680 avail 3856662528 00000100:00100000:2.0:1789303114.770985:0:26464:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3f63f480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3301:x1876219371237632:12345-0@lo:13:osp-pre-0-0.0 Request processed in 87us (799us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303114.770992:0:26464:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 307 00000100:00100000:3.0:1789303114.771235:0:3301:0:(client.c:2415:ptlrpc_check_set()) Completed RPC req@ffff960f25fbd180 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3301:1876219371237632:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1789303115.020563:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303115.020596:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303115.020634:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303115.020667:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303115.020683:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303115.020702:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303115.020722:0:3295:0:(events.c:344:request_in_callback()) peer: 12345-192.168.203.11@tcp (source: 12345-192.168.203.11@tcp) 00000100:00100000:2.0:1789303115.020749:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350639488 00000100:00100000:2.0:1789303115.020757:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350639488 00000100:00100000:2.0:1789303115.020808:0:5844:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350639104 00000100:00100000:2.0:1789303115.020811:0:5844:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350639104 00000100:00100000:2.0:1789303115.020834:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 727 00000100:00100000:2.0:1789303115.020843:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2347:x1876219350639488:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303115.020922:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf100 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:584f7d4c-528a-4a63-ad91-f53185ae8e19+36:2347:x1876219350639488:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 80us (286us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303115.020929:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 727 00000100:00100000:2.0:1789303115.020946:0:5844:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 728 00000100:00100000:2.0:1789303115.020964:0:5844:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2347:x1876219350639104:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:2.0:1789303115.020987:0:5844:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbea00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+92:2347:x1876219350639104:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 24us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1789303115.020992:0:5844:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 728 00000100:00080000:2.0:1789303115.021007:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-MDT0000 00000100:00100000:1.0:1789303115.021135:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350639232 00000100:00100000:1.0:1789303115.021140:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350639232 00000100:00100000:1.0:1789303115.021178:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350639744 00000100:00100000:1.0:1789303115.021181:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350639744 00000100:00100000:1.0:1789303115.021197:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350639360 00000100:00100000:1.0:1789303115.021200:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350639360 00000100:00100000:1.0:1789303115.021212:0:18578:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350639616 00000100:00100000:1.0:1789303115.021214:0:18578:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350639616 00000100:00100000:1.0:1789303115.021229:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 792 00000100:00100000:1.0:1789303115.021236:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350639232:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303115.021266:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd180 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350639232:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 31us (669us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303115.021272:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 792 00000100:00100000:1.0:1789303115.021284:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 793 00000100:00100000:1.0:1789303115.021290:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350639744:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303115.021311:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbf480 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+9:2347:x1876219350639744:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 21us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303115.021317:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 793 00000100:00100000:1.0:1789303115.021328:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 794 00000100:00100000:1.0:1789303115.021334:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350639360:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303115.021355:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbd500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:662dc8a6-0bb4-4da2-94bb-0ac06b41b4fa+17:2347:x1876219350639360:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 22us (653us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303115.021361:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 794 00000100:00100000:1.0:1789303115.021370:0:18578:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 795 00000100:00100000:1.0:1789303115.021377:0:18578:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350639616:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:1.0:1789303115.021396:0:18578:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f25fbfb80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:584f7d4c-528a-4a63-ad91-f53185ae8e19+10:2347:x1876219350639616:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 20us (673us total) trans 0 rc 0/0 00000100:00100000:1.0:1789303115.021401:0:18578:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 795 00000100:00080000:2.0:1789303115.021627:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0001 00000100:00080000:2.0:1789303115.021631:0:5839:0:(pinger.c:500:ping_evictor_main()) evicting all exports of obd lustre-OST0000 00000100:00100000:0.0:1789303115.022772:0:5836:0:(service.c:2301:ptlrpc_server_handle_req_in()) unwrap req x1876219350638976 00000100:00100000:0.0:1789303115.022782:0:5836:0:(service.c:2371:ptlrpc_server_handle_req_in()) got req x1876219350638976 00000100:00100000:0.0:1789303115.022838:0:5836:0:(nrs_fifo.c:152:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.203.11@tcp, seq: 710 00000100:00100000:0.0:1789303115.022847:0:5836:0:(service.c:2557:ptlrpc_server_handle_request()) Handling RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350638976:12345-192.168.203.11@tcp:400:kworker.0 00000100:00100000:0.0:1789303115.022903:0:5836:0:(service.c:2615:ptlrpc_server_handle_request()) Handled RPC req@ffff960f3353a850 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:66f3b492-6139-4ca4-a2a9-05f6a44cca16+13:2347:x1876219350638976:12345-192.168.203.11@tcp:400:kworker.0 Request processed in 58us (2350us total) trans 0 rc 0/0 00000100:00100000:0.0:1789303115.022910:0:5836:0:(nrs_fifo.c:211:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.203.11@tcp, seq: 710 00010000:00010000:3.0:1789303115.404781:0:5833:0:(ldlm_lockd.c:2804:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 69122 lines, 69122 kept, 0 dropped, 0 bad.